-- Logs begin at Mon 2022-05-23 10:02:57 UTC, end at Mon 2022-05-23 11:12:12 UTC. -- May 23 10:36:42.119803 dv-cinder-836996-25-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 10:36:42.127113 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 10:36:42 2022] *** May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: nodename: dv-cinder-836996-25-iscsi May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: machine: x86_64 May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: clock source: unix May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: pcre jit disabled May 23 10:36:42.129153 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: detected number of CPU cores: 4 May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: current working directory: / May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: detected binary path: /usr/bin/uwsgi-core May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: your processes number limit is 47680 May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: your memory page size is 4096 bytes May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: detected max file descriptor number: 2048 May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: lock engine: pthread robust mutexes May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: thunder lock: enabled May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 10:36:42.129728 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 10:36:42.174912 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: Python main interpreter initialized at 0x5583ebb379c0 May 23 10:36:42.174912 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: python threads support enabled May 23 10:36:42.174912 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: your server socket listen backlog is limited to 100 connections May 23 10:36:42.174912 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: your mercy for graceful operations on workers is 90 seconds May 23 10:36:42.175254 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: mapped 403077 bytes (393 KB) for 2 cores May 23 10:36:42.175368 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: *** Operational MODE: preforking *** May 23 10:36:42.175642 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: *** uWSGI is running in multiple interpreter mode *** May 23 10:36:42.175642 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: spawned uWSGI master process (pid: 91686) May 23 10:36:42.175847 dv-cinder-836996-25-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 10:36:42.177631 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: spawned uWSGI worker 1 (pid: 91696, cores: 1) May 23 10:36:42.177631 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: spawned uWSGI worker 2 (pid: 91697, cores: 1) May 23 10:36:42.177631 dv-cinder-836996-25-iscsi devstack@keystone.service[91686]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 10:36:47.755827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 10:36:47.755827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: from cryptography.utils import int_from_bytes May 23 10:36:47.756609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 10:36:47.756609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: from cryptography.utils import int_from_bytes May 23 10:36:47.797030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 10:36:47.797030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: from cryptography.utils import int_from_bytes May 23 10:36:47.797535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 10:36:47.797535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: from cryptography.utils import int_from_bytes May 23 10:36:47.805925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 10:36:47.806148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 10:36:47.806340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] command line args: [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 10:36:47.806607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 10:36:47.806785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ================================================================================ {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 10:36:47.807037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] admin_token = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.807283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] config_dir = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.807470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.807650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] config_source = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.807828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] debug = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.808025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_config_append = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_dir = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.809938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_file = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.810194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.810438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.810682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.810925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.811101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.812806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.813001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.813076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.813319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.813806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.814011 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] publish_errors = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.814246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.814476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.814652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.814895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.815131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.815364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] standard_threads = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.815593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.815833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] use_journal = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] use_json = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] use_stderr = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] use_syslog = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.816928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.817182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.817420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.817675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.817911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.818178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.818415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.818660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.818895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.819138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.819387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.819636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.819868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.820112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.820445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.821205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.821453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.821688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.821940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.822211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.822454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.822689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.822939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.823199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.823450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.823698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.823928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.824157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.824384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.824618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.824864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.825108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.825337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.825566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.825808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.826073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.826310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.826556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.826787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.827021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.827254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.827483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.827712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.827939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.828167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.828399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.828643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.828873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.829106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.829337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.829568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.829800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.830065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.830296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.830529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.830761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.830994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.831225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.831457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.831683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.831927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.832159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.832393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.832623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.832878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.833112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.833339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.833573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.833806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.834090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.834329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.834575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.834817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.835051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.835284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.835516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.835759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.836016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.836251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.836493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.836728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.836962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.837195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.837428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.837658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.837916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.838177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.838415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.838648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.838888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.839122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.839359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.839592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.839820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.840047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.840274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.840504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.840730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.840968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.841198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.841442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.841676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.841928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.842209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.842447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.842677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.external = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.842906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.843198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.843428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.843658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.password = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.843886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] auth.token = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.844122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.844355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.844599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.844829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.845072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.845316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.845545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.845777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.845853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 10:36:47.846039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 10:36:47.846158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.846270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] command line args: [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 10:36:47.846349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.846507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 10:36:47.846585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.846689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ================================================================================ {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 10:36:47.846767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.846947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] admin_token = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.847026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.847183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.847258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] config_dir = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.847376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.847451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.847562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] config_source = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.847681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.847765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] debug = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.847915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.847992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.848164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.848245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.848344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.848442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.848598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.848757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.848832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.848931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.849067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.849151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.849250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_config_append = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.849412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.849544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.849619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_dir = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.849688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.849762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_file = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.849866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.850116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.850205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.850276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.850346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.850495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.850574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.850744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.850827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.850924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.851025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.851103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.851273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.851389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.851493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.851597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.851770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.851875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.852004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.852109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.852247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.852372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.852479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.852558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.852722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.852805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.852966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.853047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.853198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.853284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.853429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.853672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.853748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.853903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.854138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] publish_errors = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.854217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.854294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.854395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.854472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.854577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.854706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.854827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.854905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.855067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.855220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.855304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.855384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] standard_threads = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.855521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.855606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.855765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.855843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.855964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.856126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] use_journal = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] use_json = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.856474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] use_stderr = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.856680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] use_syslog = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 10:36:47.856926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.857874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.858845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.859956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.860980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.861921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.862989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.863974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.864990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.865961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.866901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.867052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.867139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.867820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.867910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.868858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.869999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.870925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.871965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.872895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.873835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.874877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.875850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] role.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] role.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.876945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.877910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.878987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.879922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.880937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.881907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.external = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.882748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.883467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.883562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.883688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.883776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.883921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.password = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] auth.token = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.884986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.885913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.886918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.887871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.888983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.889065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.889211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.889290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.889708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.889788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890282 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.connection = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.890849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.891937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.892963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.893914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.894710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91696) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 10:36:47.894781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.895034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.895290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.895525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.895759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.895994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.896242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.896492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.896743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.896999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.897250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.897521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.897766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.898031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.898271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.898537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.898774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.899015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.899262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.899507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.899761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.899995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.900245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.900494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.900734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.900983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.901234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.901468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.901711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.901961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.902211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.902501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.902890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.903155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.903619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.903901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.904397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.904660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.905091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.905369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.905861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.906150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.906615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.906881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.907325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.907593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.908084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.908344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.908822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.909098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.909573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.909837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.910295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.910561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.911022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.911288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.911730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.911987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.912240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.912528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.912840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.913244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.913560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.913814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.914589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.914840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.915082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.915323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.915573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.915826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.916070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.916330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.916567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.916815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.917057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.917297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.917538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.917791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.918067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.918307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.918547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.918790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.919045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.919294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.919535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.919786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.920041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.920279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.920529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.920768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.921008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.921244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] role.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.921493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] role.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.921727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.921996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.922254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.922507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.922758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.923022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.923272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.923523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.923773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.924010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.924259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.924516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.924767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.925016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.925424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.925507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.925658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.925741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.925882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.926010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.926177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.926430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.926684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.926921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.927185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.927443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.927686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.927922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.928158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.928394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.928648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.928906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.929158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.929399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.929638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.930445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.930686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.930927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.931178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.931417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.931675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.931926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.932167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.932414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.932660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.932910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.933150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.933391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.933628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.933875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.934159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.934401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.934646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.934939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.935025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.935201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.connection = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.935457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.935707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.935949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.936202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.936442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.936683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.936923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.937163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.937411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.937651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.937899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.938166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.938417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.938660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.938910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.939145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.939389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.939627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 10:36:47.939800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91697) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 10:36:47.962853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.963082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.964642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.964881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.965115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.975126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.995029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:47.995266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.015854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.032965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.067012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.067244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.071272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.071511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.132960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.133101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.133200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.133322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.133405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.133542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.156518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.156758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.156895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.156983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.157891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.158052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.158271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.169535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.170067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.182415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.182650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.182872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.182953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91696) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.183844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.184061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.184277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91697) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 10:36:48.201577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.201792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 10:36:48.202168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.202378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 10:36:48.202463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.202593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 10:36:48.202674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.202903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.202985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.203109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 10:36:48.203193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.203374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.203455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.203631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.203886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.204138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.215215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.215487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.215758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.216032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.216274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.216558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.216832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.217120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.224137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.225266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.227956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.228232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.228507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.228782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.229315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.229944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.230886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.231141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.231419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.231698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.231968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.257635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.259454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.259731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.259841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.261476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.261760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.265221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.265498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.265773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.267545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.267786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.268068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.273023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.273306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.275538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.275826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.277812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.278240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.278522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.278802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.279073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.279343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.279610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.279874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.280386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.280789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.281076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.281369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.281652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.281927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.282233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.282526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.298999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.299271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.301771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.302085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.303942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.307301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.308189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.308480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.311028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.311327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.313518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.316353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.317126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.320057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.320914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.321208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.321495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.321782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.322083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.322368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.323878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.324175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.324470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.324752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.325044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.325339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.341772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.342079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.342346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.342607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.344836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.345122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.345393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.345659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.350581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.352870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.353141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.354278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.355990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.356208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.356295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.356485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.356763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.357037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.357312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.359350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.359636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.359921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.360201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.360478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.370667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.370944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.371224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.371498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.371771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.372054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.372322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.374068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.374353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.374634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.374918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.375202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.375484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.375768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.390527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.393761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.394118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.397078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.397401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.399231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.400860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.401139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.401289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.401410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.402936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.404564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.404852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.405141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.408792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.410419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.412586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.413621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.413916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.414784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.414874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.415072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.418055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.418351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.418635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.418920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.424115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.424398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.424782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.428033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.428328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.428711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.432647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91696) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.432924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.433199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.433468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.433735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.434035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.434320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.434593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.434858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.435129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.435394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.435693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.435967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.436226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91696) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.436723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91697) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 10:36:48.437008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.437292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.437573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.437847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.438155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.438436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.438729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.439013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.439297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.439570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.439834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.440098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.440372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91697) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 10:36:48.475133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5583ebb379c0 pid: 91696 (default app) May 23 10:36:48.478979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c417f-17ee-4ff5-a3c4-16f6984f934a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:48.479268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c417f-17ee-4ff5-a3c4-16f6984f934a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:48.479542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c417f-17ee-4ff5-a3c4-16f6984f934a None None] PATH_INFO: `/v3` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:48.480034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5583ebb379c0 pid: 91697 (default app) May 23 10:36:48.483844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Mon May 23 10:36:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:36:56.373582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:56.374156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:56.374156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:56.378583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:36:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:36:56.383823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:56.384203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:56.384374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:56.453519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: INFO dbcounter [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] Registered counter for database keystone May 23 10:36:56.474411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG oslo_db.sqlalchemy.engines [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91696) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 10:36:56.481190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writer thread running {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 10:36:56.655253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG passlib.handlers.bcrypt [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91696) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 10:36:56.655673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG passlib.handlers.bcrypt [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91696) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 10:36:56.710019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:36:56.712143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:36:56.832802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:36:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 10:36:56.973450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:36:57.041735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: INFO dbcounter [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] Registered counter for database keystone May 23 10:36:57.060759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG oslo_db.sqlalchemy.engines [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91697) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 10:36:57.066863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writer thread running {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 10:36:57.185270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237804a-52b1-4f3a-a3e9-1f4eedaf2a2d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:36:57.189487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:36:57.190184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:57.190471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:57.190748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:57.199334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:36:57.249858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 10:36:57.250221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29882d-bad0-482d-9302-1625342271c2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:36:57.260468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Mon May 23 10:36:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:36:57.266680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:36:57.281415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee9c41-7432-4093-95be-8b176a400423 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:36:57.285916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:36:57.287048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:57.287332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:57.287607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] PATH_INFO: `/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:57.289127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6dd918cb8494a52af5c7b9bb5c60e89)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:36:57.446903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 10:36:57.447553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d20c-d5ae-45b8-bba8-2715970c45f0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:36:57.451923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Mon May 23 10:36:57 2022] GET /identity/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:36:59.602667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a880-9dc4-4142-96d7-93b812c07d7a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:59.603219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a880-9dc4-4142-96d7-93b812c07d7a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:59.603219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a880-9dc4-4142-96d7-93b812c07d7a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:59.607344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:36:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:36:59.612750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:59.613153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:59.613308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:59.631810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:36:59.633307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:36:59.641311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:36:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 10:36:59.662937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a880-9dc4-4142-96d7-93b812c07d7a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:36:59.675279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a880-9dc4-4142-96d7-93b812c07d7a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:36:59.679419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:36:59.680038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:59.680322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:59.680594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:59.688115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:36:59.689870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86737279-7934-4055-b449-af99ce6bea2e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:36:59.698445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Mon May 23 10:36:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:36:59.703442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:36:59.712687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df34de-6aa9-47f5-89d3-fc841c2f6e8b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:36:59.716972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:36:59.717817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:36:59.718137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:36:59.718411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:36:59.719781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:36:59.721597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffa9e5-3014-4f6d-bc97-fb87cf44f93e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:36:59.723168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Mon May 23 10:36:59 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:01.963574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464ebe-e758-4f7e-992e-af8ee70e0eee None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:01.964113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464ebe-e758-4f7e-992e-af8ee70e0eee None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:01.964207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464ebe-e758-4f7e-992e-af8ee70e0eee None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:01.965495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:01.970766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:01.971171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:01.971338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:01.974345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d497-6f0d-4013-ad3e-29d12dfbae92 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:01.974633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d497-6f0d-4013-ad3e-29d12dfbae92 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:01.974911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1d497-6f0d-4013-ad3e-29d12dfbae92 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:01.976213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:01.981856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:01.982193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:01.982473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:01.989085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:01.990303 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:02.021187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG passlib.handlers.bcrypt [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91697) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 10:37:02.021614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG passlib.handlers.bcrypt [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91697) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 10:37:02.068286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:02.068427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:02.069931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:02.091347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.115346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.168041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:02.190032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.198041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aba29-45fc-4b38-8749-3bf2826e0ada None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.230843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.231579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.231867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.232148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.246148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:02.258036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:02.259149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:02.260143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:02.265154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.295228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f16b-3fcf-46a8-ba41-3f180aab2d93 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.297547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.332940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.333842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.334185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.334473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.346858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.364608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a6304-ad4d-4224-8df4-12a1d2c38af2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.397835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.399071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.399353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.399640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.400382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:02.400688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.401755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.402114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.403307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.418185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:02.419195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.500255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:02.505325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.508024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.518877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a8651-69e2-4497-88f7-9d857a173467 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.553729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.555180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.555461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.555737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.556960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.558472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.560087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:02.565499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.579226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.603473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa1353-9880-4d05-b00b-3385689b09df None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.637348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.638262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.638548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.638823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.657666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.659170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.663218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fce3e-a9cd-4bc8-b51b-be3339576fc6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:37:02.680012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:02.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:02.681621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:37:02.681621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:37:02.681621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:37:02.681621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:02.681621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 9/18] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:02.687317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.696925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.697720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.698351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.698351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.699513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.700985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.702779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 10/19] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:02.708269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:02.742683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.786095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c3ed4-35f6-45c4-82a4-45d945dbbc48 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.790949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:02.810066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d5ce5-2508-4634-bfac-df8d44cc93ce None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:02.819136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.819805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.820089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.820370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.821086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:02.821391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.822812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870358a-b98e-4aef-a534-ee483a6e75b8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.841276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 10/20] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:02.844467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:02.845136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:02.845424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:02.845702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:02.846582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.domain.id=default, user.id=eb9753922dc94eb786d2155e45d2454a) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:02.846897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:02.848423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7ec7-e006-4b43-b2be-7a30a7566a45 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:02.856684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 11/21] 10.222.0.36 () {56 vars in 1322 bytes} [Mon May 23 10:37:02 2022] GET /identity/v3/role_assignments?user.id=eb9753922dc94eb786d2155e45d2454a&scope.domain.id=default&role.id=081e04042be64de0ac1c857efdae3324 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:05.351807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6f5d-d5ad-4084-bf45-ffff36df2075 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.352866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6f5d-d5ad-4084-bf45-ffff36df2075 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.353361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b6f5d-d5ad-4084-bf45-ffff36df2075 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.355060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 11/22] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:05.361008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.361455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.361639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.387439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:05.388660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:05.484512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 12/23] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:05.516962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef2c7-2489-4367-b189-fbba8d7d2bd2 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.517563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef2c7-2489-4367-b189-fbba8d7d2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.518472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aef2c7-2489-4367-b189-fbba8d7d2bd2 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.519233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:05.520305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 12/24] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:05.526138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abf938-ebc6-4ff6-b263-15a7a21602c7 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.526711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abf938-ebc6-4ff6-b263-15a7a21602c7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.527193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abf938-ebc6-4ff6-b263-15a7a21602c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.650371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:05.679088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14713a6e-4ba5-48ae-90d6-92f21609a85f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:05.741926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-49abf938-ebc6-4ff6-b263-15a7a21602c7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:05.742636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:05.743167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-49abf938-ebc6-4ff6-b263-15a7a21602c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:05.743458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.743752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.744037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.759397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:05.775708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:05.776543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:05.777715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 13/25] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:05.784475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:05.911213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b349c67f-c954-426f-84ff-2fb685a44191 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:05.930636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 13/26] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:05.932602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ceac-4c51-48d7-a955-5ba654e70f6b None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.933088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ceac-4c51-48d7-a955-5ba654e70f6b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.933576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0ceac-4c51-48d7-a955-5ba654e70f6b None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.935183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 14/27] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:05.941336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.941901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.942397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.959472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:05.960167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:05.960576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:05.961298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:05.962392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:05.962672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:05.964364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:05.971298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:05.976892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:05.978060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:05.983502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 14/28] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:05.984350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.080656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:06.133922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 15/29] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:06.134805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.171614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de9aa-995f-4fcd-8398-536a6c623393 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.249363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.250383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.250818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.251300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.263108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.265084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.267463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d1707-4058-4844-8709-abe922ee1daa None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:37:06.284587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:06.285420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:06.286376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:37:06.286376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:37:06.286376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:37:06.286376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:06.287960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 15/30] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:06.288974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.307291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.308191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.308738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.309025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.311426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.313531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005ce3c-52ec-4e31-8937-b9679bb03c91 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.315233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 16/31] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:06 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:06.316459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572c6d-8939-4d57-b076-d2ea7debae87 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.316750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572c6d-8939-4d57-b076-d2ea7debae87 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.317037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1572c6d-8939-4d57-b076-d2ea7debae87 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.318284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 17/32] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:06.318993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572c6d-8939-4d57-b076-d2ea7debae87 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.354193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:06.387662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:06.416013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0176c7c-6a63-49fe-a297-3864a1d37b02 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.452615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.453288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.453578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.453864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.454790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.456280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3c4c1-daa0-461a-89d7-54a1d86ef0e1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.467624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1572c6d-8939-4d57-b076-d2ea7debae87 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.504579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.505251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.505542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.505821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.506607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:06.506918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.508907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.521670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 18/33] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:06.522528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.610410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 16/34] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 23 10:37:06 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:06.612175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.612473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.612765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.649130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e6989-6f74-4e6f-a0ff-461fa956bd5c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.651135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:06.651900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:06.652742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:06.717989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.718805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.719091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.719371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.720877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.722409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.724349 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 19/35] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:06.724925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.799933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 17/36] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:06.800863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:06.856548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:06.900108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d6cef-b1a5-4473-b334-cd748e0c7a29 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.934457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef980a4-7e73-43ac-96cc-2c6d30bf7e55 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:06.948183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.948867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.949156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.949438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.950192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.951922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d13042-85e4-474f-999b-a4019947fc29 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:06.973373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:06.974988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:06.975322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:06.975593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] PATH_INFO: `/v3/domains/default/users/eb9753922dc94eb786d2155e45d2454a/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:06.978217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eb9753922dc94eb786d2155e45d2454a, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:06.979819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d5936-9d28-4c5b-9874-43def5e6c2ed None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 18/37] 10.222.0.36 () {58 vars in 1361 bytes} [Mon May 23 10:37:06 2022] PUT /identity/v3/domains/default/users/eb9753922dc94eb786d2155e45d2454a/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b16c4-c05b-41d4-8a0a-43bd04851dcb None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b16c4-c05b-41d4-8a0a-43bd04851dcb None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b16c4-c05b-41d4-8a0a-43bd04851dcb None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 19/38] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:07.002620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b16c4-c05b-41d4-8a0a-43bd04851dcb None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:07.097982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 20/39] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:37:06 2022] POST /identity/v3/projects => generated 301 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:07.097982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4a80-cd39-47a8-8248-565149db307e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.097982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4a80-cd39-47a8-8248-565149db307e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.097982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bd4a80-cd39-47a8-8248-565149db307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.114815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-51bd4a80-cd39-47a8-8248-565149db307e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:07.117544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-51bd4a80-cd39-47a8-8248-565149db307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:07.142637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:07.152570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:07.161587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b16c4-c05b-41d4-8a0a-43bd04851dcb None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:07.216426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 21/40] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:07.217586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:07.217817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e5d2-412e-4710-8a0c-463d87bab9ee None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.218147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e5d2-412e-4710-8a0c-463d87bab9ee None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.218358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.218472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e5d2-412e-4710-8a0c-463d87bab9ee None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.218653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.218939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.219889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 22/41] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:07.219999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:07.221250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:07.225456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.225749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.226747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.244713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 20/42] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 23 10:37:06 2022] POST /identity/v3/roles => generated 219 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:07.250387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:07.252293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:07.256527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:07.339357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 23/43] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:07.363723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:07.381384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b182a6-c668-4996-a941-e7c243d776ea None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:07.425297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:07.425915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.426218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.426508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.427351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:07.428995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:07.448762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 21/44] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 23 10:37:07 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:07.500878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:07.515493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896cbd9-3caf-4a57-aa3d-0fb621ad3f22 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:07.558812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:07.559702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.559993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.560273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.561648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:07.563387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12339f6f-ebd7-4ddd-bf7b-4852f2b5a47b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:07.565475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 24/45] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:07.571698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:07.595390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:07.687220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acfe60-6844-4978-8e36-e85db602cabe None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:07.723363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:07.724262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:07.724550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:07.724828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:07.725543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:07.727595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d793f0f-e492-4325-bc92-3f0daf6beda9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:07.754184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 22/46] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:37:07 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:37:09.311463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9b12-eaef-4805-bb6d-2ad5b174cb61 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.311966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9b12-eaef-4805-bb6d-2ad5b174cb61 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.312078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9b12-eaef-4805-bb6d-2ad5b174cb61 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.313398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 25/47] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:09.318729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.319124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.319296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.336232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:09.337403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:09.396995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:09.402613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 23/48] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:09.425297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9b12-eaef-4805-bb6d-2ad5b174cb61 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:09.526062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9b12-eaef-4805-bb6d-2ad5b174cb61 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:09.558842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:09.559636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.559927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.560210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.571209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:09.579924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:09.580800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:09.581555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 26/49] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:09.587371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:09.684452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e21aa-a0b9-4d50-9549-43cd0802ddd6 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:09.717233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:09.717892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.718205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.718484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.719229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:09.719538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:09.720932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:09.729775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 24/50] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:09.735545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:09.751966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:09.842734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d7adf-d6bc-41cf-8ab9-23a97e2b3ba4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:09.880141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:09.881015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.881298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.881571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.883339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:09.884802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6255f2-aba2-4896-9e83-7600f9a9adaa None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:09.886468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 27/51] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:09 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:09.892067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:09.922609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f87c-fb79-47f9-904b-ab36b1efdbf5 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.922910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f87c-fb79-47f9-904b-ab36b1efdbf5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.923200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704f87c-fb79-47f9-904b-ab36b1efdbf5 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.924634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 28/52] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:09.929423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:09.929710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:09.930020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:09.946999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:09.948205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:09.954571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:10.001727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e85767-e54c-4981-8852-b8f6ba8160da None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:10.020788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 29/53] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:10.035604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:10.036388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:10.036670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:10.036940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:10.038151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:10.039680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:10.041297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 25/54] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:10.044041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:10.046626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:10.056075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:10.141042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71acbb5-5b4a-4b2f-af5f-064a8473e8dd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:10.144806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d9fc6-9f3f-49c4-ac50-45b5e0210ba1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:10.171726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:10.173255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:10.174074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:10.174363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:10.174636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:10.175816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:10.177275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:10.179642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 30/55] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:10.182653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:10.183300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:10.183586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:10.183866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:10.184701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.domain.id=default, user.id=eb9753922dc94eb786d2155e45d2454a) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:10.184937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:10.185062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:10.186598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0acf-044b-4017-b429-94644bed7901 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:10.192780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 26/56] 10.222.0.36 () {56 vars in 1322 bytes} [Mon May 23 10:37:10 2022] GET /identity/v3/role_assignments?user.id=eb9753922dc94eb786d2155e45d2454a&scope.domain.id=default&role.id=081e04042be64de0ac1c857efdae3324 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:10.269003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:10.291776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8108c-fb75-4532-b8a0-53b4e638ae88 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:10.328656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:10.329344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:10.329636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:10.329922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:10.330695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:10.332207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74a31-8315-4de6-947b-77a0ff007661 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:10.358739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 31/57] 10.222.0.36 () {60 vars in 1140 bytes} [Mon May 23 10:37:10 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:37:12.535943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc75ded-73ec-4927-bd6f-e49fb2981c88 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:12.536448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc75ded-73ec-4927-bd6f-e49fb2981c88 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:12.536529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc75ded-73ec-4927-bd6f-e49fb2981c88 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:12.537897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 27/58] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:12.543048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:12.543433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:12.543620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:12.560601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:12.561799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:12.626603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 32/59] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:12.650720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc75ded-73ec-4927-bd6f-e49fb2981c88 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:12.751533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc75ded-73ec-4927-bd6f-e49fb2981c88 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:12.783661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:12.784447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:12.784739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:12.785024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:12.786284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:12.787705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bf9d5-6d78-4868-af21-2e43d51f650e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:12.789806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 28/60] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:12.795177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:12.852319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:12.892040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847a58-5604-4d44-b98b-8cedc9e45877 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:12.924394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:12.925056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:12.925346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:12.925628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:12.926371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:12.927837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1ae31-5dc9-45b4-b1d4-9685d3da0cfa None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:12.951956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 33/61] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:12 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:16.465770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b449f2-b90c-437e-8437-5b5c0b20a3eb None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.465770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b449f2-b90c-437e-8437-5b5c0b20a3eb None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.465770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b449f2-b90c-437e-8437-5b5c0b20a3eb None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.465770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 29/62] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:16.472110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dad0a-3e93-4ebe-a57c-5d2b2e173043 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.472951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dad0a-3e93-4ebe-a57c-5d2b2e173043 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.473462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dad0a-3e93-4ebe-a57c-5d2b2e173043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.491178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9e1dad0a-3e93-4ebe-a57c-5d2b2e173043 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:16.493271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9e1dad0a-3e93-4ebe-a57c-5d2b2e173043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:16.576657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 34/63] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:16.591325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514be4-9d4a-43d7-a1c7-72df176071db None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.591633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514be4-9d4a-43d7-a1c7-72df176071db None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.591902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514be4-9d4a-43d7-a1c7-72df176071db None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.593354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 30/64] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:16.609265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c514be4-9d4a-43d7-a1c7-72df176071db None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:16.611549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb83f4-5c58-4977-95a8-108bdcce87d5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.611880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb83f4-5c58-4977-95a8-108bdcce87d5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.612198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb83f4-5c58-4977-95a8-108bdcce87d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.636050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-39eb83f4-5c58-4977-95a8-108bdcce87d5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:16.637257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-39eb83f4-5c58-4977-95a8-108bdcce87d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:16.714317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:16.721652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:16.766974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 35/65] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:16.783746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c514be4-9d4a-43d7-a1c7-72df176071db None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:16.800455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163187bb-d455-4685-af48-1c72831bf2e6 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.801072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163187bb-d455-4685-af48-1c72831bf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.801554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163187bb-d455-4685-af48-1c72831bf2e6 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.803174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 36/66] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:16.804886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163187bb-d455-4685-af48-1c72831bf2e6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:16.820267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:16.821069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.821355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.821641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.832118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-2196e75a-0041-422a-9d7a-d4505ec9e342 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:16.840297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:16.841157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:16.841917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 31/67] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:16.842929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f0f74-1bd3-47ef-b406-918b4f54efb7 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.843181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f0f74-1bd3-47ef-b406-918b4f54efb7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.843464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f0f74-1bd3-47ef-b406-918b4f54efb7 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.844877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 32/68] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:16.845839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.846326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.846676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.863956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:16.865156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:16.916801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163187bb-d455-4685-af48-1c72831bf2e6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:16.935812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 33/69] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:16.936676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:16.966763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:16.967564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.967853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.968131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:16.978930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-11347941-4d4b-4090-ab90-9ed4a133de18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:16.986475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:16.987551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:16.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:16.990470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:16.992619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 37/70] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:16.994031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaeddb-9ef0-4370-879d-58b7aef04005 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:16.994326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaeddb-9ef0-4370-879d-58b7aef04005 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:16.994610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaeddb-9ef0-4370-879d-58b7aef04005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.010505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-99eaeddb-9ef0-4370-879d-58b7aef04005 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:17.011694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-99eaeddb-9ef0-4370-879d-58b7aef04005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:17.039283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc678f2-63f6-452f-b9ff-1f55bb792cc1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.041479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.071850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.072524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.072809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.073088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.073828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:17.074163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.075582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.079410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 38/71] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:17.080733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ccc08-cc63-43c7-b08d-e91496fc7ae5 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.081029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ccc08-cc63-43c7-b08d-e91496fc7ae5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.081309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ccc08-cc63-43c7-b08d-e91496fc7ae5 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.087246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 39/72] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:17.087246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ccc08-cc63-43c7-b08d-e91496fc7ae5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.088789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 34/73] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.089619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.187186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7bfd2-0f28-45d0-b63f-2d6365b01219 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.191250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ccc08-cc63-43c7-b08d-e91496fc7ae5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.201240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.221698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.222391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.222677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.222953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.223691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:17.224027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.225415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41a93b-ece8-4334-9299-9349140cd263 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.226521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.227337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.227630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.227910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.233538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 35/74] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.235057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.235355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.235638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.243476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:17.251722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:17.252556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:17.253577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 40/75] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:17.254488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:17.254569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.254777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:17.283776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.322683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 36/76] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:17.323544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.358319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d059-d7b5-4a60-84f3-cf59500bb674 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.391246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2034239-b346-449f-9607-f88318481b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.392115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034239-b346-449f-9607-f88318481b67 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.392399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034239-b346-449f-9607-f88318481b67 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.392676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2034239-b346-449f-9607-f88318481b67 None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.407659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2034239-b346-449f-9607-f88318481b67 None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.409135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2034239-b346-449f-9607-f88318481b67 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.410822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 41/77] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.412683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2034239-b346-449f-9607-f88318481b67 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.424724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa9b92-01c3-451b-861b-839a8f65a6b7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.431996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.458535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.459323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.459602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.459881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.461663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.463157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.464761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 37/78] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.469866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.496386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.512822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2034239-b346-449f-9607-f88318481b67 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.546346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.547217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.547510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.547786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.548983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.550454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.552073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 42/79] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.552832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.569270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561186d-aecc-4c22-9b54-8bd58fe4543b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.603098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.603694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.603974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.604250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.604992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:17.605300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.606715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.614384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 38/80] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:17.615196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.650404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226654b7-37ad-4776-b52d-0c2157e2a865 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.651119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.684006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.684806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.685097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.685380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.695988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:17.703625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:17.704565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 43/81] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:17.710266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.714919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668bb4a-4a01-4e90-9064-cdc6a18f3f4f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.724465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.748177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.748837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.749321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.749528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.807704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c944088-1cdd-4247-8f08-8d8d937dedcc None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:17.814940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.816616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.840003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:17.840658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:17.840946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:17.841225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:17.841923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:17.843396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:17.850400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.863277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 44/82] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:17 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:17.864049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.928567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 39/83] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:17.928567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:17.941366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:17.982554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f8581-6cfc-446d-a47f-68497579c520 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.021774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.022917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.023388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.023741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.041971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a71b0e-0d53-47f6-b4a1-47ca44910fd1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.080182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.081064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.081351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.081640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.083434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.085868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.086787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 40/84] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.087482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.091564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.111518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.113033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.195280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.218826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185905b0-9abe-46ac-8cca-597cfb20d8dc None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.239443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 45/85] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:18.240686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.278311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.279105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.279474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.279825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.280653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:18.281043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.282600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.292463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 41/86] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.293336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.354241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f212a-b654-4bcb-bfcd-c6228e5b79b6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.373115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.389893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.390900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.391260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.391613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.392907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.394520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.396546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 46/87] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.402445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.415592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85f89a-7ad2-481c-b770-ac517385e853 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.438702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.456899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.457769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.458786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.459222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.509728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6287e-80cf-4b21-9663-e5ca8ff88341 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.551407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.551646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.553006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.553197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.553623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.554106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.555752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.558354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.560578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 47/88] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.561587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.630138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.665978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27ac83-0b45-4ee5-9d32-0c1b3677c318 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.668348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.671458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 42/89] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:18.672740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.701801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.702862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.703155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.703431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.705220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.706955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.709126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 48/90] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.709906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.784323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb0c53-dda7-4be8-98d8-c7a9a8b38d4e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.821042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.821977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.822499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.822880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.823826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:18.824293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.826125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.832213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 43/91] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=63df59ac99514b3983611a6b633ec83f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.838546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.842771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58736c3d-6b0c-4e89-bba8-cda229efdc2d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:18.898179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:18.898179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:18.898179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:18.898179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:18.898612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=eb9753922dc94eb786d2155e45d2454a) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:18.898612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:18.899868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:18.917590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 49/92] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/role_assignments?user.id=eb9753922dc94eb786d2155e45d2454a&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:18.922752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:18.924276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:18.991878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:19.100304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d41c5-09d4-4922-82df-679b68a3bb51 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:19.158549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110673d-fa1a-407b-a286-bdcfd8c32674 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:19.185577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:19.186290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:19.186585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:19.186867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:19.226833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:19.227725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:19.228017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:19.228301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:19.232958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:19.237133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:19.237819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 44/93] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:19.246090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:19.360868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:19.362411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:19.432631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:19.450300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:19.454966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b869982-c7ad-486b-ae76-9fa0c77b9599 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:19.522824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:19.524877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:19.525169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:19.525458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:19.526335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:19.526656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:19.528117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:19.543055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 45/94] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:19 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:19.553562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 50/95] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:19.560001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:19.697783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec7c2b-94f7-481b-8e34-b5404fbde909 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:19.739031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:19.739918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:19.740203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:19.740481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] PATH_INFO: `/v3/projects/1a1d357ebc954d028127bffcfb221d08` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:19.742318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] RBAC: Authorizing `identity:get_project(project_id=1a1d357ebc954d028127bffcfb221d08)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:19.743820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f5028-58f8-47a5-bdd5-3a03232e8250 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:19.745637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 46/96] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:19 2022] GET /identity/v3/projects/1a1d357ebc954d028127bffcfb221d08 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:19.752440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:19.886586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261165b6-4711-4ff0-abcf-b95d9d7f9dca None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:19.948546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:19.949219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:19.949509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:19.949791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:19.950667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=1a1d357ebc954d028127bffcfb221d08, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:19.950985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:19.952425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb579-efc0-4b73-8031-640455be30ef None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:19.955607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:19.964755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 51/97] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:19 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=1a1d357ebc954d028127bffcfb221d08&role.id=63df59ac99514b3983611a6b633ec83f => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:20.672722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0861f4-3da1-4ae6-b3a0-6f7c4f6c3aa3 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:20.673241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0861f4-3da1-4ae6-b3a0-6f7c4f6c3aa3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:20.673346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0861f4-3da1-4ae6-b3a0-6f7c4f6c3aa3 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:20.674906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 47/98] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:20.680760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:20.681635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:20.682165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:20.706416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:20.707953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:20.713885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7a34-5d1b-4d08-970c-cc66078301c9 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:20.714446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7a34-5d1b-4d08-970c-cc66078301c9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:20.714914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d7a34-5d1b-4d08-970c-cc66078301c9 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:20.716497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 48/99] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:20.721358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:20.721860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:20.722362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:20.750621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:20.753530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:20.815015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:20.827236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 52/100] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:20.870881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:20.903466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 49/101] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:20.926887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:21.042509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d391b-7135-4e42-b2e4-7c156e452391 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.056284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c38b13-f5ce-4e1f-bec5-5f8731b56a44 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.092021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.093170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.102345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.102874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.104332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:21.106045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:21.114945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 50/102] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:21.119190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:21.144996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.146193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.148325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.149628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.154207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:21.172054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:21.173612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:21.179820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:21.180643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:21.181420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 53/103] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:21.187275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:21.344895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d7d6-ec74-4a1b-873f-fa99b6c67bca None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.352889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac69871-ca56-4f27-837a-ce7d9daa0cc6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.397245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.398894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.399399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.399865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.400821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:21.401322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:21.403042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:21.416041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.416925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.417360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.417780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.418685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:21.420318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:21.423883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 54/104] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:21.429696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:21.478308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 51/105] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:37:21 2022] POST /identity/v3/projects => generated 302 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:21.530420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:21.573885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248ba3b-c821-4140-8383-3af709a76956 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.626555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.627446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.627738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.628028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] PATH_INFO: `/v3/users/dc298b62e3944f39b3ce37a6865eb1ce` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.655644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] RBAC: Authorizing `identity:get_user(user_id=dc298b62e3944f39b3ce37a6865eb1ce)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:21.657147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:21.658953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 55/106] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:21 2022] GET /identity/v3/users/dc298b62e3944f39b3ce37a6865eb1ce => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:21.665424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:21.807476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:21.833849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0c9a1-0116-48af-9700-edecb1299dd2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:21.884301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:21.884962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:21.885249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:21.885531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:21.980563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:21.982101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5bbf-c1fd-4092-b395-81c5d2f2086b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:22.086098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:22.108612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 52/107] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:22.114216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:22.202841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52129cd-0362-4b94-9e6e-b33b2a9e02fa None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.203169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52129cd-0362-4b94-9e6e-b33b2a9e02fa None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.203422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52129cd-0362-4b94-9e6e-b33b2a9e02fa None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.204874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 53/108] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:22.211409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.211727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.211975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.247081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:22.248187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:22.333773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03b1f-3c8e-4bc0-8aac-8a3ecfb5c427 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:22.356785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 54/109] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:22.357730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:22.391734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:22.397064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:22.398493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.398780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.399143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.402721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:22.404216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671da8c-dddf-42d0-9f6e-93181441f4b5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:22.406295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 56/110] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:22 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:22.429906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc3eb-f976-4914-bda3-7ff818889370 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.430247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc3eb-f976-4914-bda3-7ff818889370 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.430532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffc3eb-f976-4914-bda3-7ff818889370 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.431920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 57/111] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:22.432619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc3eb-f976-4914-bda3-7ff818889370 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:22.589463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93366aa7-74bd-477c-af6f-e7c6483be8de None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:22.591476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffc3eb-f976-4914-bda3-7ff818889370 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:22.610242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:22.654329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:22.654999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.655288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.655568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.656414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=dc298b62e3944f39b3ce37a6865eb1ce) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:22.656729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:22.658223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc910e35-91be-4447-a65e-12fd52de0ffb None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:22.661221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:22.662039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.662326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.662596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.671664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 58/112] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:22 2022] GET /identity/v3/role_assignments?user.id=dc298b62e3944f39b3ce37a6865eb1ce&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=4262ac95da0148d29d3d912e069159dc => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:22.672758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:22.673050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:22.673332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:22.684065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:22.692035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:22.693013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:22.693809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:22.706042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 55/113] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:22.710871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:22.712172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:22.713225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:22.816124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:22.874058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 59/114] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:22.924918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.002084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328813e2-0d24-4c12-b5c6-ac2513ce4876 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.080788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.081470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.081764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.082078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.082839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:23.083153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.084737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:23.104786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 56/115] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:23.119669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542acae-fdc6-4d6f-ae3d-8e58257b28b4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.127950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.183759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.199332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.200165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.200458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.200742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.212734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-3cf2ec28-5183-42aa-9fa7-2080e8eb33cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:23.225218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:23.226675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:23.227407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 60/116] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:23.228589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ced-6c92-4bd9-89bc-42e38abf68aa None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.228754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ced-6c92-4bd9-89bc-42e38abf68aa None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.228998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9ced-6c92-4bd9-89bc-42e38abf68aa None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.230456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 61/117] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:23.231510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb500-d334-4407-b89f-8516828a6788 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.231808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb500-d334-4407-b89f-8516828a6788 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.232092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb500-d334-4407-b89f-8516828a6788 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.239989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 62/118] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:23.239989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fb500-d334-4407-b89f-8516828a6788 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.283658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.307985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34cc11-d330-4b38-bf67-0c1d74c5c885 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.349527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.350803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.351277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.351727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.353131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.354809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de9761-32f6-4535-ae6d-f6f2d80f845d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:23.356699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 57/119] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:23.358164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.358675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.359172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.367051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fb500-d334-4407-b89f-8516828a6788 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.389083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:23.390534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:23.415363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.416863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.417433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.417888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.419011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:23.419802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.421887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbfdde-1b15-4932-8166-14eae48ee9cc None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:23.433664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 63/120] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:23.435191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.435658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.436117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.456402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:23.458287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:23.467281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 58/121] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:23.468250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.510333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.544109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 64/122] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:23.545048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.579019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13645c5c-1220-43b0-bbe9-10369f90c117 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.583583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.623437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.624758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.625143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.625522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.626932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.629138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:23.636344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 59/123] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:23.636344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.669280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cbcf4-0cce-4896-8a38-cc649209214e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.710562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.711769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.712972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.713451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.715721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.717595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:23.720348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 65/124] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:23.725100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.757893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.762760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ddd6a-0991-42df-81f1-64ab90d29815 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.800615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.801764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.802278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.802782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.814850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:23.824856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:23.825691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:23.826474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:23.828865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 60/125] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:23.830035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.830719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:23.840367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae664790-6b9e-4e52-8e97-1df1ae19c990 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.876707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.877513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.877856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.878161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.890199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:23.898485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:23.899342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:23.900106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 66/126] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:23.905705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:23.950415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3030-1a68-4dea-ad4a-bb335a65a699 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:23.991933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:23.993994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:23.994491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:23.994986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/55f99bba48794e39be6d96b2d2956952/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:23.997291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, user_id=55f99bba48794e39be6d96b2d2956952, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:23.999120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.016318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 61/127] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:23 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/55f99bba48794e39be6d96b2d2956952/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:24.018721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.027794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a01f-2212-4dab-9448-33ff69f24480 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.047803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.073443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.074597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.075045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.075510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.077115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.079005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.081247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 67/128] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:23 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:24.082437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.121121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.209700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4806ba1-6998-4eb0-8ff9-095cb18f0221 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.253997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.253997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.253997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.253997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.254456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:24.254456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.254456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.263493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 68/129] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:24.264491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.273031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96feabc5-6bec-488c-9de7-29d32f68033d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.316768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.317464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.317758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.318172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.318939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:24.319284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.320789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.329984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 62/130] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:24.330850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.360625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.380541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62937da0-d7cf-4b20-b846-63a02c2006e7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.417510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.418515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.418800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.419078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/eb9753922dc94eb786d2155e45d2454a/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.420751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.422063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, user_id=eb9753922dc94eb786d2155e45d2454a, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.423651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.442688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6aba8a-1887-40b7-bddc-68c59fbe8891 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.451121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 69/131] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:24 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/eb9753922dc94eb786d2155e45d2454a/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:24.451965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.485754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423978ab-874a-4c78-8776-793b4063099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.486719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423978ab-874a-4c78-8776-793b4063099d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.487018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423978ab-874a-4c78-8776-793b4063099d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.487306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423978ab-874a-4c78-8776-793b4063099d None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.488538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423978ab-874a-4c78-8776-793b4063099d None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.490304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423978ab-874a-4c78-8776-793b4063099d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.492630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 63/132] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:24.498306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423978ab-874a-4c78-8776-793b4063099d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.614869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423978ab-874a-4c78-8776-793b4063099d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.625294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9cba6c-2516-482f-83d5-eb9e201f8983 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.653268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.654293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.654584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.654870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] PATH_INFO: `/v3/projects/1a1d357ebc954d028127bffcfb221d08` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.656203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] RBAC: Authorizing `identity:get_project(project_id=1a1d357ebc954d028127bffcfb221d08)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.657823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9a8e9-cc4d-480e-a21a-70a6dec79f93 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.659829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 64/133] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/projects/1a1d357ebc954d028127bffcfb221d08 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:24.661025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0057e-f122-4c77-aa44-f9dab09b515f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.661323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0057e-f122-4c77-aa44-f9dab09b515f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.661609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0057e-f122-4c77-aa44-f9dab09b515f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.668199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 65/134] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:24.669008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0057e-f122-4c77-aa44-f9dab09b515f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.690138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.705276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.706232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.706527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.706816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.708401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.712169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ddab93-236e-400d-9261-6e87918ac2ac None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.718144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 70/135] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:24.719625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.719923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.720212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.751753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:24.752936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:24.756523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:24.821932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0057e-f122-4c77-aa44-f9dab09b515f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:24.844633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 71/136] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:24.846121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.875568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:24.876477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:24.876763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:24.877040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] PATH_INFO: `/v3/projects/1a1d357ebc954d028127bffcfb221d08/users/55f99bba48794e39be6d96b2d2956952/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:24.879065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] RBAC: Authorizing `identity:create_grant(project_id=1a1d357ebc954d028127bffcfb221d08, user_id=55f99bba48794e39be6d96b2d2956952, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:24.880540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:24.913885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 66/137] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:24 2022] PUT /identity/v3/projects/1a1d357ebc954d028127bffcfb221d08/users/55f99bba48794e39be6d96b2d2956952/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:24.914764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:24.983798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf434d9-ae6f-47a4-a0b5-f4d8ef82b9e8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:25.032432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:25.033659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.034147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.034605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.036715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:25.038553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:25.040633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 72/138] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:25.046675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:25.066017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:25.124582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:25.188138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d10f-11d6-4405-a85f-ae725f119fae None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:25.213995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67622e-0cb6-4156-93bc-67c5a8187287 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:25.262948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:25.263821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.264112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.264394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/55f99bba48794e39be6d96b2d2956952/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.266999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, user_id=55f99bba48794e39be6d96b2d2956952, role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:25.268514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40aa3ed-ec9e-4f12-8e43-d6055d66d943 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:25.275909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:25.276795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.277363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.277657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.279507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:25.281308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:25.283662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 67/139] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:25.292423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 73/140] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:25 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/55f99bba48794e39be6d96b2d2956952/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:25.293383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:25.433167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81beca-9954-456c-b3f3-614fae515d1e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:25.483094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:25.483782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.484128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.484417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.485149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:25.486653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:25.526701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:25.530957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78936370-8974-4608-b623-f61eeb481748 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.531281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78936370-8974-4608-b623-f61eeb481748 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.531956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78936370-8974-4608-b623-f61eeb481748 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.533128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 74/141] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:25.539065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 68/142] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:25 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:25.544082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.544375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.544662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.568353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:25.569703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:25.642863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:25.651007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 75/143] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:25.676589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:25.800904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78bc4-e45a-4359-9b50-3e2d519394a4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:25.848015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:25.848833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:25.849125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:25.849410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:25.866098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:25.873839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:25.874720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:25.875468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 69/144] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:25.882509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:25.994828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ac06a-c363-4bd3-ae31-bb744d446ba1 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:26.037378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:26.038695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:26.038980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:26.039253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:26.040004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:26.040306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:26.041705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:26.056642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 76/145] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:26.062603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:26.170739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:26.235687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5ea3-ffe5-4b80-93d6-c1f57ace5df8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:26.294805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:26.295687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:26.295972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:26.296246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] PATH_INFO: `/v3/users/dc298b62e3944f39b3ce37a6865eb1ce` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:26.298731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] RBAC: Authorizing `identity:get_user(user_id=dc298b62e3944f39b3ce37a6865eb1ce)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:26.300449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:26.302358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 70/146] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:26 2022] GET /identity/v3/users/dc298b62e3944f39b3ce37a6865eb1ce => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:26.307861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:26.401736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:26.447412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ce4f-09d6-4e71-b16c-85c3606ba670 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:26.486896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:26.487796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:26.488083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:26.488366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:26.489618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:26.493091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db412d85-a120-42de-a7f9-cf69fc76f413 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:26.494934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 77/147] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:26 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:26.500776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:26.636456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ed014-ec33-4364-98d9-80526eb69487 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:26.663526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:26.671523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:26.672962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:26.673645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:26.674205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/dc298b62e3944f39b3ce37a6865eb1ce/roles/4262ac95da0148d29d3d912e069159dc` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:26.676741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, user_id=dc298b62e3944f39b3ce37a6865eb1ce, role_id=4262ac95da0148d29d3d912e069159dc)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:26.678247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339f391-d499-475a-8741-58e76bba1de9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:26.702692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 71/148] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:26 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/users/dc298b62e3944f39b3ce37a6865eb1ce/roles/4262ac95da0148d29d3d912e069159dc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:27.229097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823d2c-760c-4c73-925d-470106bb4199 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:27.229458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823d2c-760c-4c73-925d-470106bb4199 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:27.229674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823d2c-760c-4c73-925d-470106bb4199 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:27.232920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 78/149] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:27.236757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:27.243507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:27.244018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:27.269024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:27.270628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:27.351177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 72/150] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:27.394776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74823d2c-760c-4c73-925d-470106bb4199 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:27.515322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74823d2c-760c-4c73-925d-470106bb4199 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:27.543635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:27.551797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:27.552614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:27.552901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:27.553214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:27.565053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:27.572997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:27.573837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:27.574619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 79/151] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:27.581315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:27.719338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a0ce9-1c80-45af-bf53-fae3ee26690f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:27.739225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:27.765571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:27.766669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:27.767172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:27.767644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:27.768600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:27.769105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:27.770789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:27.786699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 73/152] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:27.792044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:27.921053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce574d90-8fb8-48ad-9f7b-2c69973a3108 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:27.956505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:27.957393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:27.957681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:27.957975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:27.959228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:27.961284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62f795-59c6-4c63-9698-b54cf4699dd3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:27.963124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 80/153] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:27 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:27.968812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:28.087458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a684622-229b-4323-a9d0-a92fb0836806 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:28.133603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:28.134315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.134603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.134887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.189889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:28.214913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:28.216403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:28.321109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cefde-de53-4b32-956d-0631dddeb753 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.321412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cefde-de53-4b32-956d-0631dddeb753 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.321681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cefde-de53-4b32-956d-0631dddeb753 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.323160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 81/154] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:28.328426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd63e-a570-45ea-bd07-ed5a451ae46f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.328717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd63e-a570-45ea-bd07-ed5a451ae46f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.329002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fcd63e-a570-45ea-bd07-ed5a451ae46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.352321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-15fcd63e-a570-45ea-bd07-ed5a451ae46f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:28.353421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-15fcd63e-a570-45ea-bd07-ed5a451ae46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:28.384610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 74/155] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:28.390816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:28.492098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:28.526410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:28.526595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 82/156] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:28.527575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7823b50-4e3e-41cc-86d7-00b3e328eaa5 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.529980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7823b50-4e3e-41cc-86d7-00b3e328eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.529980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7823b50-4e3e-41cc-86d7-00b3e328eaa5 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.529980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 83/157] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:28.543719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.544006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.544285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.558825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb8270-c91d-440c-a480-c7c4ad4413cc None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:28.583154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:28.584384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:28.617145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:28.617934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.618393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.618679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.620057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:28.621668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:28.627857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 75/158] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:28 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:28.628710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:28.768402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 84/159] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:28.769289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:28.791102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139f60c-0a37-406c-89da-c7a64c6291fe None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:28.850252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:28.851627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:28.851915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:28.852195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:28.871316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:28.879754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:28.880585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:28.881599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 76/160] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:28.882343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:28.904949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:29.001692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad90a0-567a-4d8c-9d35-14b685b936c6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.036986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:29.088058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.088734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.089025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.089308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.090185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:29.090507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:29.091971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:29.104095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9608-8220-4c71-9133-c33bf8c91313 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.105202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 85/161] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:28 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=63df59ac99514b3983611a6b633ec83f => generated 608 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:29.105853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:29.202917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.203669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.203960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.204242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.236530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:29.244339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:29.245207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:29.246042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 77/162] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:29.259047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:29.376204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951cec5-0964-4cb3-8daa-042ee1e189b5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.457122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:29.478799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.479499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.479792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.480075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.480846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:29.481159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:29.482606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:29.501571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 86/163] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:29.506788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:29.521705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393effec-c0a4-4cd9-a5e3-be5bd8e5ec7b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.567471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.568524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.569273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.570411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.571527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:29.573798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:29.575446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:29.599716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 78/164] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:29.604945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:29.607076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:29.731655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235872fa-4695-4c8b-912e-e0f68f0b2910 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.808269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.809759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.811239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.812556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] PATH_INFO: `/v3/users/eb9753922dc94eb786d2155e45d2454a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.815065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] RBAC: Authorizing `identity:get_user(user_id=eb9753922dc94eb786d2155e45d2454a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:29.817314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77527e0-7681-47f5-a83b-59c1fbcd4d74 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:29.819075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:29.820979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 87/165] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/users/eb9753922dc94eb786d2155e45d2454a => generated 247 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:29.829432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:29.882638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:29.900020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:29.900916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:29.901201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:29.901478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:29.906732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:29.908224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:29.910309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 79/166] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:29.916331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:30.025895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41483bcf-3c90-4b37-a3cf-dec9d05ab6ff None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.042078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:30.089068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.090121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.090704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.091183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.111810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dde70-f2c6-45ab-afa3-08bc0eb587e6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.168008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.168668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.168950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.169227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.186995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:30.188503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76998a34-8c49-4228-a024-eef9cb7ddc85 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:30.319256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 88/167] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:30.320958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc6b2d-a0b6-4ace-ade4-fe4d769c6470 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.321450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc6b2d-a0b6-4ace-ade4-fe4d769c6470 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.322071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc6b2d-a0b6-4ace-ade4-fe4d769c6470 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.323679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 89/168] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:30.331170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.331711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.332205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.362513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:30.364273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:30.365350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:30.375300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:30.375848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:30.376910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:30.500024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 80/169] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:30.501111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:30.519388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 90/170] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:30.520974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:30.695896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:30.698464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0bc61-967f-44f1-95dd-8712b0e2a9f0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.717465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6d5e1-f98f-4dd9-b335-a6ee28729c38 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.765755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.766651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.766937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.767216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.769031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:30.770570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:30.773184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 81/171] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:30 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:30.773805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:30.791381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.792280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.792567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.792848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] PATH_INFO: `/v3/projects/1a1d357ebc954d028127bffcfb221d08` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.794184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1a1d357ebc954d028127bffcfb221d08)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:30.795709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427cd6-0bd8-4914-868c-dbd16ef702b8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:30.797543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 91/172] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:30 2022] GET /identity/v3/projects/1a1d357ebc954d028127bffcfb221d08 => generated 312 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:30.798907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aaf2d-3cc6-490e-aa00-684165b79cea None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.799198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aaf2d-3cc6-490e-aa00-684165b79cea None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.799479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3aaf2d-3cc6-490e-aa00-684165b79cea None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.800807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 92/173] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:30.801539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3aaf2d-3cc6-490e-aa00-684165b79cea None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:30.828640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:30.918840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce72e-f683-40aa-9fd1-0c7f2d580e7f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.944680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3aaf2d-3cc6-490e-aa00-684165b79cea None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:30.954731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.955482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.955771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.956053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.967339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a6912cb2-5a7b-4f26-93e6-3f8ff736aa65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:30.976192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:30.977130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:30.978159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 82/174] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:30.983270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:30.983942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.984116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb3c-5b66-4db3-9038-de4360cb428d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:30.984292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.984465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb3c-5b66-4db3-9038-de4360cb428d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:30.984648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.984989 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624bb3c-5b66-4db3-9038-de4360cb428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:30.985395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=eb9753922dc94eb786d2155e45d2454a) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:30.985709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:30.987496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:30.995525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 93/175] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:30 2022] GET /identity/v3/role_assignments?user.id=eb9753922dc94eb786d2155e45d2454a&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:30.996315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.003302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e624bb3c-5b66-4db3-9038-de4360cb428d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:31.004583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e624bb3c-5b66-4db3-9038-de4360cb428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:31.029978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.083760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 83/176] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:31.085292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1a4d-f914-4fd2-a5ec-cffddef66230 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.085654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1a4d-f914-4fd2-a5ec-cffddef66230 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.086038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc1a4d-f914-4fd2-a5ec-cffddef66230 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.087474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 84/177] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:31.088275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc1a4d-f914-4fd2-a5ec-cffddef66230 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.091153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.147458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56145198-97c2-4bc1-82e6-6a03cc6ab27b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.186429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.187100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.187871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.188320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.189243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=1a1d357ebc954d028127bffcfb221d08, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:31.189555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:31.191048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac8def-0d1a-4e47-bb91-fc25063e9099 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:31.198803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 94/178] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:30 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=1a1d357ebc954d028127bffcfb221d08&role.id=63df59ac99514b3983611a6b633ec83f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:31.200020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b260f-a43d-471c-9b48-fd5ab6685174 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.200129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b260f-a43d-471c-9b48-fd5ab6685174 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.200391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b260f-a43d-471c-9b48-fd5ab6685174 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.205028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 95/179] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:31.205028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db84c1-df61-42ca-bcc6-c5cb04b865ad None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.205028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db84c1-df61-42ca-bcc6-c5cb04b865ad None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.205028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db84c1-df61-42ca-bcc6-c5cb04b865ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.211033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc1a4d-f914-4fd2-a5ec-cffddef66230 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.222175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-82db84c1-df61-42ca-bcc6-c5cb04b865ad None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:31.223704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-82db84c1-df61-42ca-bcc6-c5cb04b865ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:31.260217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.260902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.261187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.261458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.262300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:31.262612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:31.264053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:31.273876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 85/180] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:31.275662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.298312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.320800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 96/181] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:31.322225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada90b-6aee-4009-ba00-037bc85945f2 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.322523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada90b-6aee-4009-ba00-037bc85945f2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.322804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ada90b-6aee-4009-ba00-037bc85945f2 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.324491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 97/182] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:31.325471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.325847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.326190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.345701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:31.347033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:31.402873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b31cf1-a662-434e-9062-4b28213a04ee None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.417068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.427209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 98/183] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:31.428313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.442537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.443497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.443787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.444085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.457110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-eff60c36-f77a-40d9-8f86-c72986dae452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:31.463033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:31.463926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:31.464851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 86/184] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:31.465597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.465715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.466679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.484970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:31.486242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:31.540884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721c9b0-0899-48f2-b0ae-a5f5e257e4d7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.554899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 87/185] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:31.555751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.565211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.587606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.588484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.588766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.589043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] PATH_INFO: `/v3/users/55f99bba48794e39be6d96b2d2956952` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.590265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] RBAC: Authorizing `identity:get_user(user_id=55f99bba48794e39be6d96b2d2956952)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:31.591733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:31.593366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 99/186] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/users/55f99bba48794e39be6d96b2d2956952 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:31.594207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.655628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.656513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bd855-a433-4f85-acf1-8d7411663f54 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.691274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.692069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.692359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.692639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.692878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90701fcb-ad96-4138-98c1-5cd6fc11b05f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.704208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:31.711955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:31.712857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 88/187] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:31.718769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.726133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.726948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.727231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.727508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.737516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:31.746210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:31.747212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:31.747998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 100/188] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:31.749646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.781560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.816295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2a553-8b74-4afa-bbdb-d862c67e45c2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.846693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e7020-2ad9-49d4-b240-d88033de669b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.848688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.849350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.849637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.849918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.851255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:31.851561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:31.852942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:31.860931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 89/189] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:31.861546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.870176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.882246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.883250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.883534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.883811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:31.884995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:31.886494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:31.888131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 101/190] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:31.888890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:31.958973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee53857-8b9d-44be-9f40-41a5ef8f6383 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.985788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959b0f-8a29-4edc-8f0e-55c5ff46b8eb None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:31.988511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:31.994752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:31.995409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:31.995701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:31.995983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.019888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.020559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.020851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.021131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.021876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:32.022741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.024125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.032194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 102/191] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.032977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.067769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.069251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.087735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.131490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf8ab9-75e7-40ba-a3b9-191dd0b11e63 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.166253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.166925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.167211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.167309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 90/192] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:32.167489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.168112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.168272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:32.168538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.169928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.179803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 103/193] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.181772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.197315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.266841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97709-c6f0-45e7-8063-76f7aef4d1b9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.296930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccec7a-bd7c-4a93-951e-342664799bd0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.299697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.300614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.300939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.301240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.314720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.316053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.316165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.323179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 91/194] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.323179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.337290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.338293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.338758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.339211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.339996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.341450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.359802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 104/195] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:32 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:32.360954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.425807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.439634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa03c77-5601-4d65-8871-698a2353d7e7 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.479698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.480589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.480872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.481147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] PATH_INFO: `/v3/users/dc298b62e3944f39b3ce37a6865eb1ce` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.482431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] RBAC: Authorizing `identity:get_user(user_id=dc298b62e3944f39b3ce37a6865eb1ce)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.483928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.485810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 92/196] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/users/dc298b62e3944f39b3ce37a6865eb1ce => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.491048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.504809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3bbc58-a39c-4385-a160-b4c765750449 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.549787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.551542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.551542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.551937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.554604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.556575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.559872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 105/197] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.560892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.615465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.617733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c3f5-0367-44b2-909b-8496c76f265a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.662029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.663730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.664243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.664692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.666664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.668300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.670168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 93/198] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:32.671124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.702717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95e58c-0ad2-4160-9e8c-1b44a86ffeed None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.703464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.750635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.751598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.752061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.752514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.802067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5efd-ca57-4e7d-8302-d88c948ae715 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:32.845431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.846957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.851111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:32.851780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:32.852063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:32.852340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:32.896363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:32.948165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:32.950061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:32.966204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 106/199] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:32.970830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:32.983231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.086157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e003c53-640d-4d61-addd-fbe4da4ce6aa None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.090905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 94/200] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:33.091764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.121616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.122393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.122677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.122953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.123910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=55f99bba48794e39be6d96b2d2956952) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:33.124251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.125866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.132204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 107/201] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:32 2022] GET /identity/v3/role_assignments?user.id=55f99bba48794e39be6d96b2d2956952&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.133008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.136665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.206449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab40c2-841d-4a76-8226-327c7b9a4c91 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.280884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.286926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.287595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.287907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.288223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.322412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436cf16-743c-4739-a5a4-22b0684198bc None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.375899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.377348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.397415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.398312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.398596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.398876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.400143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.401565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.403307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 108/202] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.404080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.486390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.508263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 95/203] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:33.509133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.531114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13caba3b-8dd4-4469-b066-dd8bd1d6b8d0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.534569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.573198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.574256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.574537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.574815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.576011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.577715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.579453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 109/204] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.580315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.614806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5757e-bec3-44c6-8622-1e23ebafdd40 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.654590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.655184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.655454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.655736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.656589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=054d56f9484c43bebc1aa29c96161ea7) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:33.656910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.658408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.664996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 96/205] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/role_assignments?user.id=054d56f9484c43bebc1aa29c96161ea7&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.665880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.692653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd7db-cd15-44ba-a1cf-7baef90ed85c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.731518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.732568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.732854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.733203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.734635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.736318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.738621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 110/206] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.744656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:33.757034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.766839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:33.788977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28562050-4c5f-4ee2-bf66-d7ae42f83f1b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.849678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.850933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.851252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.851504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.852370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d, user.id=dc298b62e3944f39b3ce37a6865eb1ce) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:33.852689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.854239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9151a-94a5-43ec-80bd-957df7e6b9d2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.861983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab73b22-3fc7-44e6-b589-c7d69ff2ec5a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:33.869845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 97/207] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/role_assignments?user.id=dc298b62e3944f39b3ce37a6865eb1ce&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=4262ac95da0148d29d3d912e069159dc => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:33.929487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:33.930611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:33.931152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:33.931658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:33.932802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=054d56f9484c43bebc1aa29c96161ea7) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:33.933365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:33.935165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43c021-de75-4ce7-a919-327920bf9d97 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:33.942436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 111/208] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:33 2022] GET /identity/v3/role_assignments?user.id=054d56f9484c43bebc1aa29c96161ea7&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:34.849478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2199ee-843d-4b04-8205-01af3fe589f1 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:34.850027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2199ee-843d-4b04-8205-01af3fe589f1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:34.850106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2199ee-843d-4b04-8205-01af3fe589f1 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:34.852298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 98/209] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:34.857780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08c9d-e068-4126-a678-9a7bbea45433 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:34.858205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08c9d-e068-4126-a678-9a7bbea45433 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:34.858390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b08c9d-e068-4126-a678-9a7bbea45433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:34.875672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a4b08c9d-e068-4126-a678-9a7bbea45433 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:34.876847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a4b08c9d-e068-4126-a678-9a7bbea45433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:34.950060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 112/210] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:34.968937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2199ee-843d-4b04-8205-01af3fe589f1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.007857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d34a-0bdc-4919-bdff-4255a2c63a14 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.008241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d34a-0bdc-4919-bdff-4255a2c63a14 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.008600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d34a-0bdc-4919-bdff-4255a2c63a14 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.010479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 113/211] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:35.015650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.016015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.016309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.021967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.039043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:35.040249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:35.082319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2199ee-843d-4b04-8205-01af3fe589f1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.114269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 114/212] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:35.126245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.127110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.127398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.127681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.138725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:35.140575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:35.148934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:35.149891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:35.150757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:35.155734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 99/213] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:35.158517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.178190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.273940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059339f8-c54a-4633-bfc6-8408dc441ef0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.286237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689e197-678a-4f9a-9554-19324fb62725 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.315054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.322905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.323047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.323733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.324218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.324329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.324618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.324880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.325290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.325632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:35.326032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:35.326931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:35.327744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:35.328663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:35.331508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 115/214] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:35.337366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.341567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 100/215] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:35.351922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.449352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.455909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0bf7-e013-48ca-9ae7-62a11f8484bd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.477681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a79066-46d5-4063-a966-5408b2c07453 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.503573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.504489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.505158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.505629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.507324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:35.508800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:35.515653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.517543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.518598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.519512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] PATH_INFO: `/v3/users/b16890c7e6ce41518f9a25876ac980e4` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.533066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 116/216] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:35 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:37:35.535287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] RBAC: Authorizing `identity:get_user(user_id=b16890c7e6ce41518f9a25876ac980e4)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:35.537013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:35.544337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 101/217] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:35 2022] GET /identity/v3/users/b16890c7e6ce41518f9a25876ac980e4 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:35.546794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:35.618615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.670418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c2a97-57d8-4223-bbb9-b6be332963e4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:35.708352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:35.709248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:35.710280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:35.710718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:35.784060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:35.785867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:35.844090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:35.898055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 117/218] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:35.900762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.038776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c4492-d76b-4b93-8304-17d955dfca8d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:36.074454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:36.080415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:36.081316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.081605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.081886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.086902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:36.088453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db088b-7c6e-408b-8b48-fbf66f7bed0f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:36.090634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 102/219] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:35 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:36.097062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.197559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e257-7e52-4a29-9636-d19852cf5ded None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.197851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e257-7e52-4a29-9636-d19852cf5ded None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.198171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7e257-7e52-4a29-9636-d19852cf5ded None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.199726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 103/220] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:36.204754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.205046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.205327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.234991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:36.236211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:36.251550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d251b-caa0-4ce1-a44b-251bdd59dab8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:36.291489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:36.292621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.293089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.293563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.294628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=b16890c7e6ce41518f9a25876ac980e4) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:36.295125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:36.296756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96302899-fbe6-49cf-bef0-11adfcfd7653 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:36.303170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 118/221] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/role_assignments?user.id=b16890c7e6ce41518f9a25876ac980e4&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=63df59ac99514b3983611a6b633ec83f => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:36.318552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604e0-e45b-45a6-9577-a7b30677c312 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.318990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604e0-e45b-45a6-9577-a7b30677c312 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.319849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53604e0-e45b-45a6-9577-a7b30677c312 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.321385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 119/222] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:36.327435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.327734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.328019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.345297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:36.346568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:36.370623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:36.424363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 120/223] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:36.435463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 104/224] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:36.447938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.472335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.563004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c853df1-9062-4dad-9430-b54cbb2afcfe None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:36.608226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:36.609055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.609612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.609909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.622412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:36.631430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:36.632816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:36.633609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:36.638277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 121/225] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:36.640381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.654411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:36.688275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0bf1d-d493-45c3-92ec-66b0306c699d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:36.753566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac399d-0dc1-43bc-be8b-8ef33b04c248 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:36.789574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:36.792956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:36.794048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.794487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.794984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.795888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:36.796347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:36.797903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:36.808149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 122/226] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:36.814163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.819278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:36.820091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:36.820379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:36.820655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:36.862575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:36.870784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:36.871867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:36.872878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:36.878421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 105/227] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:36.883668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:36.914288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:36.956641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a770b9a-cc0b-49ae-9154-25d874eab79c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.003712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.004602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.004893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.005174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.006454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.008058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.010320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 123/228] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:37.016077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:37.133799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21ee64-7684-44bf-a784-b8cb94603ba5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.172398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a9f19-2e59-42d0-b8e2-e840c2d60a86 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.172906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.173397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.173695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.174282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.175509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.176990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.179755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 124/229] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:37 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:37.185250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:37.198312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:37.296030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385565-02e9-4058-b6c9-153176fae985 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.311501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.312198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.312482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.312760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.313519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:37.313825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.315242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.331701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.332757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.333166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.333460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/users/054d56f9484c43bebc1aa29c96161ea7/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.336320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, user_id=054d56f9484c43bebc1aa29c96161ea7, role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.337253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 106/230] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:37.337794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.343069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:37.355060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 125/231] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:37 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/users/054d56f9484c43bebc1aa29c96161ea7/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:37:37.451267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:37.516029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c0534-d4f9-4e1f-9104-e53dd04a6ea6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.563408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.564353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.564642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.564921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.566225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.567791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.571008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 107/232] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:37 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:37.576038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:37.650741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:37.687396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3750-a2ef-4a72-b5d4-9e7be6ca6718 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.725416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.726697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.727185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.727665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.729068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.730803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce8000-5582-4a07-8ee3-1dc587d29271 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.732755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 126/233] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:37 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:37.738444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:37.879182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4c7a4-55ff-47a7-974e-3f849b4c5036 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:37.908568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:37.931576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:37.932477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:37.932761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:37.933096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/users/054d56f9484c43bebc1aa29c96161ea7/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:37.935170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, user_id=054d56f9484c43bebc1aa29c96161ea7, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:37.936628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f494d56-5a65-4db0-a90a-3aaf0da1d817 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:37.962062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 108/234] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:37 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/users/054d56f9484c43bebc1aa29c96161ea7/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:38.283021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89ddb-ea4e-4feb-9dd8-5aeb7b45d30a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.283523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89ddb-ea4e-4feb-9dd8-5aeb7b45d30a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.283621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89ddb-ea4e-4feb-9dd8-5aeb7b45d30a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.284976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 127/235] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:38.291074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.291500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.291692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.317859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:38.319117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:38.404612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 109/236] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:38.442585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:38.442978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86c7f-5796-48f8-a398-917b144fb839 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.443270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86c7f-5796-48f8-a398-917b144fb839 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.443554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce86c7f-5796-48f8-a398-917b144fb839 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.445282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 128/237] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:38.473567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.474203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.474685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.507124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:38.508639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:38.546939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec1462-8927-43a6-9aab-88001a7101ab None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:38.562687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:38.583315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:38.585126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.585408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.585779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.587304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:38.588848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:38.590854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 110/238] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:38.597189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:38.667009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 129/239] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:38.720570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2ce57-3868-42f3-8333-e87dac5e8186 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:38.747090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:38.765712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:38.766334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.766617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.766890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:38.767685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:38.769480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:38.798904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 111/240] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:38 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:38.885260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:38.934373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731d62-5dba-4d4a-89a0-24d789374494 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:38.986165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:38.986988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:38.987276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:38.987556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.004837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:39.012869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:39.013794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:39.015297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 130/241] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:39.022138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.077594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:39.154214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9502-9070-4a85-a2f2-d052f254ecf9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:39.191598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:39.192613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.193091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.193560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.194560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] RBAC: Adding query filter params (name=reader) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:39.195079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:39.196720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b676bf-6129-4ff0-b2b4-6f73965ac77b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:39.210101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 112/242] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:39.211684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.302134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba3ad-c140-4741-866f-0ad09674851c None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.302640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba3ad-c140-4741-866f-0ad09674851c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.302872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeba3ad-c140-4741-866f-0ad09674851c None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.306947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 113/243] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:39.318369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.318672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.318985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.327366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a69cb-15f0-4c97-8ee2-ccb378b28c16 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:39.346557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:39.347779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:39.367972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:39.382390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:39.383699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.384186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.384670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] PATH_INFO: `/v3/users/692ea49295234e5fb00dcb7d2f8e0f49` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.405490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] RBAC: Authorizing `identity:get_user(user_id=692ea49295234e5fb00dcb7d2f8e0f49)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:39.407483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:39.409608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 131/244] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/users/692ea49295234e5fb00dcb7d2f8e0f49 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:39.415959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.434968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 114/245] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:39.457824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.493930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:39.553035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650ed78-ac23-45a8-ba8d-4eb124c87907 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:39.580060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1040f-80e8-430f-bf15-c4b637dfdc7c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:39.598121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:39.599036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.599326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.600021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.626866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:39.627674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.627957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.628235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.645035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:39.653447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:39.654369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:39.655222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 115/246] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:39.661643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.674236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:39.696769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:39.698849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:39.813462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 132/247] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:39.819934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.829530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:39.847089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e486d-23c2-4c1d-a06f-e8b1dd9dc19a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:39.882783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:39.883459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:39.883746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:39.884026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:39.889986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:39.889986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:39.889986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:39.897694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 116/248] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:39.905981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:39.989644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:40.036152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b8ac-12c3-421f-9437-2bcc361e5dd8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.079631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2693-1bd4-4697-8bd8-0241e00bf2d0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.119123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.120788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.120788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.120788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.126987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:40.128538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:40.130797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 133/249] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:40.139078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.154669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.155980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.156457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.156920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] PATH_INFO: `/v3/users/b16890c7e6ce41518f9a25876ac980e4` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.158414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] RBAC: Authorizing `identity:get_user(user_id=b16890c7e6ce41518f9a25876ac980e4)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:40.160095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:40.161995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 117/250] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:39 2022] GET /identity/v3/users/b16890c7e6ce41518f9a25876ac980e4 => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:40.167582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.218317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:40.356232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063357a7-afec-47d9-923a-4a9315cf14f6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.360440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b368-6f39-451a-95c7-bfc329775282 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.414848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.415718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.415999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.416273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.421984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:40.421984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4412d0-dbd6-47dc-86e2-f85aeb8e472f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:40.422371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 118/251] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:40 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:40.424027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b76c-0d2b-4108-8808-0c79f2fbd0a5 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.424482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b76c-0d2b-4108-8808-0c79f2fbd0a5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.425007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b76c-0d2b-4108-8808-0c79f2fbd0a5 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.425581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=692ea49295234e5fb00dcb7d2f8e0f49) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:40.433974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458644ab-57e5-449e-afd0-26cd971c81c5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:40.436007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 119/252] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:40 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:40.436827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b76c-0d2b-4108-8808-0c79f2fbd0a5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.441292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 134/253] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:40 2022] GET /identity/v3/role_assignments?user.id=692ea49295234e5fb00dcb7d2f8e0f49&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=4262ac95da0148d29d3d912e069159dc => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:40.442741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.443037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.443316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.475095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:40.476308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:40.494737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:40.596780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 135/254] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:40.640105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.678382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:40.686533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b76c-0d2b-4108-8808-0c79f2fbd0a5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.724538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.725499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.725790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.726123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/users/b16890c7e6ce41518f9a25876ac980e4/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.728184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, user_id=b16890c7e6ce41518f9a25876ac980e4, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:40.729722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f6b44-ce32-46ed-955d-e7b505a41232 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:40.749013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 120/255] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:40 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/users/b16890c7e6ce41518f9a25876ac980e4/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:40.750394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2481-9bea-435c-b31d-f44887a4b710 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.750685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2481-9bea-435c-b31d-f44887a4b710 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.750968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2481-9bea-435c-b31d-f44887a4b710 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.752411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 121/256] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:40.758583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.758879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.759155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.788402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:40.790280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:40.845153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd8587-9d6d-4402-a9f8-7b7adbcb7753 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:40.920892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 122/257] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:40.927423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:40.928855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:40.929138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:40.929411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:40.945500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.946767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:40.957434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:40.958422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:40.959218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 136/258] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:40.964497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:40.977286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.033049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.060387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83456a79-60b9-4422-889d-ae6ae7ea913f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.098863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.099942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.100378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.100836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.111870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:41.120450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:41.121523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:41.122529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:41.123797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d7744-afea-47b9-826b-57f079000f04 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.129971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 123/259] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:41.130845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:41.187377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.188272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.188703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.189121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.190067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:41.190513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.192051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.205157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 137/260] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:41.210617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:41.236335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf0fd2-58c3-4fa2-aefc-57588e51a6bb None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.272781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.273784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.274371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.274870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.275825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:41.276336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.277974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.286848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 124/261] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:41.289974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.293368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:41.341796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.408571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5953b4-2ce6-4f0c-9f27-eeb39d112a39 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.447664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9b28f-9b18-4cc9-b4e5-dd511d49d1ce None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.479255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.480168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.480462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.480742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.486679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.488455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.491221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 138/262] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:41 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:41.496078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:41.511640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.512732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.513157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.513674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] PATH_INFO: `/v3/users/054d56f9484c43bebc1aa29c96161ea7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.518379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] RBAC: Authorizing `identity:get_user(user_id=054d56f9484c43bebc1aa29c96161ea7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.520132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.522165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 125/263] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:41 2022] GET /identity/v3/users/054d56f9484c43bebc1aa29c96161ea7 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:41.527793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:41.634439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.673181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bd1e6-c7a9-48d8-abf0-21801c352b8d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.720156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2e72-95e6-4e21-b087-8f607127f21a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:41.743258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.743933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.744227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.744511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.762306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:41.793197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:41.794265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:41.794870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:41.795340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:41.836225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.837645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.902636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:41.904142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:41.968041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:42.009276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 126/264] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:42.015726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:42.024137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 139/265] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:42.027788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:42.086397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:42.208853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42cd5e8-bfab-476b-9ab2-115cc7cc1960 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:42.288628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:42.289519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.289808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.290116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.291353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:42.292832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:42.294896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 140/266] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:42.300983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:42.310796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d2b058-ccfc-4f0b-b48a-7a12ec477346 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:42.418726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:42.419864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.420287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.420727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.426873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:42.429161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:42.431166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 127/267] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:42.443042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:42.500299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a38f1-b0fe-4dc6-877c-66253fdb967a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:42.504822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:42.519153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:42.572643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:42.573225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.573508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.573800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.574651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=054d56f9484c43bebc1aa29c96161ea7) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:42.574961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:42.576471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0fb33-dd0e-4976-8cd9-c59fd97a1d74 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:42.589427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 141/268] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/role_assignments?user.id=054d56f9484c43bebc1aa29c96161ea7&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:42.609308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5284-ad15-4cfc-9a84-3592ea0f7a1b None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.609610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5284-ad15-4cfc-9a84-3592ea0f7a1b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.609878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f5284-ad15-4cfc-9a84-3592ea0f7a1b None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.613029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 142/269] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:42.626827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.627382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.627805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.652261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:42.653727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:42.677860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f7d46-3e0f-42c1-9b65-e4104f9e20b5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:42.718502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:42.719543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.720032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.720510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.721581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=054d56f9484c43bebc1aa29c96161ea7) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:42.722197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:42.724047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384572cb-1366-44b3-8fd5-f1b6c6cf2a85 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:42.732844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 128/270] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/role_assignments?user.id=054d56f9484c43bebc1aa29c96161ea7&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:42.735114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b4f7-ea25-4b09-9698-fbe0296055e8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.735617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b4f7-ea25-4b09-9698-fbe0296055e8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.736121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b4f7-ea25-4b09-9698-fbe0296055e8 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.737878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 129/271] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:42.748225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:42.748852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:42.749334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:42.789481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:42.791835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:42.833488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 143/272] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:42.895523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:42.919866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 130/273] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:42.971599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.019268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.094416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.096887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103ce27-9489-4dd7-8add-5bf3459cab19 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.147285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604a3ec-5e69-48fc-9f73-cc577b83988e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.149469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.150308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.150592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.150935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.153711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:43.155231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:43.157385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 144/274] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:43.165462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.193228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.196002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.196394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.196764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.210415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:43.218467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:43.219359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:43.220120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:43.224058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 131/275] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:43.230790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.302902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce64a-f0ef-4b30-884b-da8a68001550 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.331597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.341625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.342561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.343103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.343519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.344524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:43.346231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:43.349781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a5e26-d982-4868-b069-76e9b0cd1fef None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.372583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 145/276] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:37:43 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:37:43.393403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.394423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.394779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.395118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.395906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:43.396257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:43.398441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:43.413220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 132/277] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:43.416253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.446220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.534336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47378119-d22d-448e-baba-5d6fd0ee3e0c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.580206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.581118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.581406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.581686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] PATH_INFO: `/v3/users/9974ab349b674f6cb650a99abb758d1c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.599288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] RBAC: Authorizing `identity:get_user(user_id=9974ab349b674f6cb650a99abb758d1c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:43.600775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc678ba-674b-4dd3-a76e-2a1896057881 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:43.603075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 146/278] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:43 2022] GET /identity/v3/users/9974ab349b674f6cb650a99abb758d1c => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:43.608947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.719817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2c396-561d-46df-ad21-3a5c2de5ddf8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:43.757190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:43.757386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500be14-316e-43a9-8725-06feaf697d2a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.757655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500be14-316e-43a9-8725-06feaf697d2a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.758176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500be14-316e-43a9-8725-06feaf697d2a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.758404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.758692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.758970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.759822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, user.id=9974ab349b674f6cb650a99abb758d1c, scope.system=all) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:43.760265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 147/279] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:43.760378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:43.761729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3323367-c737-45b5-b6bd-bf224b104be8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:43.766234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.766789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.767071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.768115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 133/280] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 23 10:37:43 2022] GET /identity/v3/role_assignments?user.id=9974ab349b674f6cb650a99abb758d1c&scope.system=all&role.id=63df59ac99514b3983611a6b633ec83f => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:43.791810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:43.793044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:43.800826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433654f6-2f62-499e-8e77-054fcd9f9451 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.801141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433654f6-2f62-499e-8e77-054fcd9f9451 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.801387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433654f6-2f62-499e-8e77-054fcd9f9451 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.803498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 134/281] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:43.809156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:43.809437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:43.809721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:43.835317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:43.836774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:43.848097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.886173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 148/282] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:43.909657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:43.926377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:43.932408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 135/283] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:43.964817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.092509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280afd88-9133-46e3-bebb-97506ac1e3ab None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.132537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c841143-cfd7-452e-babf-783a3a2adf9d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.134105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.135149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.135605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.136045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.148178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:44.156216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:44.157090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:44.157866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:44.160311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 136/284] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:44.166229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.170383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:44.211979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.212783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.213061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.213323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.243316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:44.257655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:44.258596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:44.259381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:44.268309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 149/285] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:44.270561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.341260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc067372-bbb1-4536-b342-93a27f4b805b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.384096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:44.403770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.404378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.404660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.404936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.405688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:44.406016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:44.407459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:44.417160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286f1fb-1601-4797-b493-0c9bf3accc12 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.426262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 137/286] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:44.432077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.469882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.470590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.470885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.471163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.471921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:44.472237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:44.478102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:44.491744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 150/287] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:44.497641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.578635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:44.605993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27fbf-66a9-44ac-9348-1225c2d87b90 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.637446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:44.645422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1566063-ba24-4905-a7c3-11760485e064 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.657474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.658675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.659169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.659663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] PATH_INFO: `/v3/users/692ea49295234e5fb00dcb7d2f8e0f49` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.662946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] RBAC: Authorizing `identity:get_user(user_id=692ea49295234e5fb00dcb7d2f8e0f49)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:44.664629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:44.667425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 151/288] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/users/692ea49295234e5fb00dcb7d2f8e0f49 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:44.673384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.709087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.710872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.711360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.711833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] PATH_INFO: `/v3/users/b16890c7e6ce41518f9a25876ac980e4` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.714470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] RBAC: Authorizing `identity:get_user(user_id=b16890c7e6ce41518f9a25876ac980e4)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:44.716411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:44.720562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 138/289] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/users/b16890c7e6ce41518f9a25876ac980e4 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:44.724719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.846712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce3f3f-4a3e-43c7-97b2-2b061c7d3945 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.903753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347041-61df-43aa-8743-4ff32b323d7a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:44.915591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.916728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.917215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.917692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:44.920769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:44.922547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:44.924884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 152/290] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:44.930565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:44.966319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:44.974245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:44.978156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:44.979156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:44.980446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:45.064474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:45.127774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:45.130713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:45.133063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1d2d-5487-4d4b-8e3d-32259b3e9567 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:45.204609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:45.205836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:45.206347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:45.206845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/users/692ea49295234e5fb00dcb7d2f8e0f49/roles/4262ac95da0148d29d3d912e069159dc` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:45.210501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, user_id=692ea49295234e5fb00dcb7d2f8e0f49, role_id=4262ac95da0148d29d3d912e069159dc)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:45.212160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:45.248012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 153/291] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:37:44 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/users/692ea49295234e5fb00dcb7d2f8e0f49/roles/4262ac95da0148d29d3d912e069159dc => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:37:45.309680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 139/292] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:45.315602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:45.446476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417eb8f-d469-4a35-8238-59ed242bc642 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:45.475061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:45.485510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:45.486756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:45.487267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:45.487743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:45.489172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:45.491505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e68415-e007-494c-bb55-ba7ab013a160 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:45.493740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 154/293] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:45 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:45.499971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:45.533553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:45.611394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a47cc-8c7a-4625-9215-2ecbe1d57706 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:45.646948 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:45.648379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:45.648909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:45.649398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:45.650474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=b16890c7e6ce41518f9a25876ac980e4) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:45.651042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:45.652722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1133ac-3989-4c07-8633-78c751042ad4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:45.659689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 140/294] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:45 2022] GET /identity/v3/role_assignments?user.id=b16890c7e6ce41518f9a25876ac980e4&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=63df59ac99514b3983611a6b633ec83f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:46.125389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbd23e-0bdf-4ac5-a882-57de2cbdce8d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.125936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbd23e-0bdf-4ac5-a882-57de2cbdce8d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.125936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbd23e-0bdf-4ac5-a882-57de2cbdce8d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.127553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 155/295] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:46.133015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96cf45-b665-4ccb-99f8-f3a91df5440d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.133312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96cf45-b665-4ccb-99f8-f3a91df5440d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.133599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96cf45-b665-4ccb-99f8-f3a91df5440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.152457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7b96cf45-b665-4ccb-99f8-f3a91df5440d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:46.153663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7b96cf45-b665-4ccb-99f8-f3a91df5440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:46.223331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 141/296] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:46.248500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbd23e-0bdf-4ac5-a882-57de2cbdce8d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.312334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734715b5-dce2-40ab-bb8c-50c38a7752f3 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.312911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734715b5-dce2-40ab-bb8c-50c38a7752f3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.313373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734715b5-dce2-40ab-bb8c-50c38a7752f3 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.315248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 142/297] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:46.320617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e364f-6495-4585-b27e-7aa8ee54d88e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.321176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e364f-6495-4585-b27e-7aa8ee54d88e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.321657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e364f-6495-4585-b27e-7aa8ee54d88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.353731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-803e364f-6495-4585-b27e-7aa8ee54d88e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:46.355014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-803e364f-6495-4585-b27e-7aa8ee54d88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:46.378744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:46.383258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbd23e-0bdf-4ac5-a882-57de2cbdce8d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.424311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.425291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.425577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.425854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.432972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 143/298] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:46.437156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:46.439856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2318bde-2fbd-4d89-9787-56e162dbf641 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.440385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2318bde-2fbd-4d89-9787-56e162dbf641 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.440864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2318bde-2fbd-4d89-9787-56e162dbf641 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.442436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 144/299] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:46.445545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:46.446748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:46.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:46.448146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.448146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.448427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.450162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 156/300] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:46.455569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.474176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:46.476158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:46.516844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:46.554111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 145/301] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:46.555296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.566287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56899-a9f3-4b62-b59b-b8bb3936d4a9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.604662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.605612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.605612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.605881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.606683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:46.606994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:46.608387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:46.617762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 157/302] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:46.618421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.657927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:46.675307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d053d-de18-4f67-ad5a-07d23a5520ce None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.712436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.713246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.713528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.713804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.723586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cba4d5-fcc9-4e43-9a04-9518779167f2 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.724216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:46.731947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:46.732821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:46.733589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 146/303] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:46.734286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.745626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:46.765266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.766080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.766360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.766622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.767788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:46.769232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:46.776072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 158/304] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:46.776799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.840401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786854a1-be2d-43d5-88d9-17549131b299 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.875352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.876212 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.876497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.876909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] PATH_INFO: `/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.879544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6f27e-12b5-4ecb-b9ab-7347e559fdc2 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:46.890840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] RBAC: Authorizing `identity:get_user(user_id=2cd69cfa2d9044fc90dea5e0ce3235bf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:46.892322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:46.894209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 147/305] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:46.894780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.899369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:46.920613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:46.921124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:46.921411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:46.921694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:46.922477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:46.922788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:46.924162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:46.932481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 159/306] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:46.933092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:46.985241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:47.006659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc9455-3d0c-4367-ba64-255a8233ed1a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:47.040316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19db077-7487-4cb0-a9cd-dbed1ed78610 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:47.042087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:47.042771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.043054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.043327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] PATH_INFO: `/v3/groups` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.044026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:47.045484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:47.069983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 148/307] 10.222.0.36 () {60 vars in 1134 bytes} [Mon May 23 10:37:46 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:47.069983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:47.088294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:47.089061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.089444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.089795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.090871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, user.id=2cd69cfa2d9044fc90dea5e0ce3235bf, scope.system=all) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:47.091393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:47.093005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:47.101783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 160/308] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 23 10:37:46 2022] GET /identity/v3/role_assignments?user.id=2cd69cfa2d9044fc90dea5e0ce3235bf&scope.system=all&role.id=4262ac95da0148d29d3d912e069159dc => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:47.153208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:47.202183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9f86d-fed8-4894-bfe0-ee832085f656 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:47.243389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:47.244496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.244573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.244838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] PATH_INFO: `/v3/users/9974ab349b674f6cb650a99abb758d1c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.246274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9974ab349b674f6cb650a99abb758d1c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:47.247803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d497e79-597b-4a7d-9b7e-6a7e769454c8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:47.249908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 149/309] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:47 2022] GET /identity/v3/users/9974ab349b674f6cb650a99abb758d1c => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:47.255779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:47.369497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69bdc-a1b3-4e90-90ae-6655b0ca23ac None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:47.383223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:47.405660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:47.406562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.406852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.407135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] PATH_INFO: `/v3/system/users/9974ab349b674f6cb650a99abb758d1c/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.408661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9974ab349b674f6cb650a99abb758d1c, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:47.410083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ebb3-98ce-49ee-bd79-5ea0b48c64dc None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:47.420887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 161/310] 10.222.0.36 () {58 vars in 1334 bytes} [Mon May 23 10:37:47 2022] PUT /identity/v3/system/users/9974ab349b674f6cb650a99abb758d1c/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:47.707073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781405-4f2a-4531-a504-afb50c0245ef None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.707586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781405-4f2a-4531-a504-afb50c0245ef None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.707660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25781405-4f2a-4531-a504-afb50c0245ef None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.709293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 150/311] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:47.714702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.715231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.715714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.733071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:47.734619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:47.809975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 162/312] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:47.829912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25781405-4f2a-4531-a504-afb50c0245ef None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:47.942128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25781405-4f2a-4531-a504-afb50c0245ef None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:47.976306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:47.982685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:47.983739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:47.984179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:47.984886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:47.997792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:48.005899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:48.006991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:48.007979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:48.010311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 151/313] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:48.016534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:48.117995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b06a6-d8d9-46f5-8f77-97bfa2f2db7f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:48.126212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:48.155110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:48.155999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:48.156441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:48.156870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:48.157761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:48.158268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:48.159806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:48.169263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 163/314] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:48.174929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:48.284906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf2825-36bd-4ca4-9e25-cbf621ce738d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:48.325179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:48.326441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:48.326930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:48.327404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] PATH_INFO: `/v3/users/692ea49295234e5fb00dcb7d2f8e0f49` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:48.328845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] RBAC: Authorizing `identity:get_user(user_id=692ea49295234e5fb00dcb7d2f8e0f49)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:48.330564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:48.332709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 152/315] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:48 2022] GET /identity/v3/users/692ea49295234e5fb00dcb7d2f8e0f49 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:48.338389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:48.443098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7ef1-ae64-4b1c-a3c5-14ae260b2e4a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:48.477936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:48.478728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:48.479015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:48.479297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:48.514366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:48.548327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:48.549853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:48.652499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 164/316] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:48.657979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:48.720030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:48.764287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a36c2-cf9e-483e-8764-a8a1c53aa426 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:48.799831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:48.800925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:48.801449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:48.801876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:48.803281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:48.804933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0371b4-634e-4aff-b49c-2260d6595319 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:48.807125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 153/317] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:48 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:48.817981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:48.881436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:48.916284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b94bb-3f5b-439b-8e3b-5417e86ec980 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:48.951877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:48.952545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:48.952835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:48.953105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:48.953964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, scope.project.id=12cb24e441a941148f32f0504181b73b, user.id=692ea49295234e5fb00dcb7d2f8e0f49) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:48.954286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:48.955737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7ca71-6c9d-45ef-b135-70d328fbfa32 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:48.962886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 165/318] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:37:48 2022] GET /identity/v3/role_assignments?user.id=692ea49295234e5fb00dcb7d2f8e0f49&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=4262ac95da0148d29d3d912e069159dc => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:49.411456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861fc0a-8e20-49d5-84ac-2bddfd564d51 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.411742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861fc0a-8e20-49d5-84ac-2bddfd564d51 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.412722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861fc0a-8e20-49d5-84ac-2bddfd564d51 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.414268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 154/319] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:49.419340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.419770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.419894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.437167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:49.438372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:49.461911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa95a7-c574-46a2-bce7-e6f4cbc0dad8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.462366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa95a7-c574-46a2-bce7-e6f4cbc0dad8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.462547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa95a7-c574-46a2-bce7-e6f4cbc0dad8 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.464102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 155/320] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:49.469833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.470168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.470456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.490138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:49.491951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:49.514068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 166/321] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:49.534753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:49.570007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 156/322] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:49.590212 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:49.600444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:49.643125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b0ba4-59a6-4bd6-88e7-67ae8c03adc3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:49.650639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:49.680164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:49.680984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.681269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.681548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.682754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:49.684212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ed127-c95c-4962-aa89-7743cee38b86 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:49.685846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 167/323] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:37:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:49.691511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e82e1-b5e6-446c-84ef-8685ee3437c6 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.691864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e82e1-b5e6-446c-84ef-8685ee3437c6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.692145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e82e1-b5e6-446c-84ef-8685ee3437c6 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.693986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 168/324] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:49.694103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c47623-f653-4e2c-9960-b19346f12b8b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:49.694184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e82e1-b5e6-446c-84ef-8685ee3437c6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:49.727755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:49.728550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.728833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.729104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.739027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-dbc47d0d-9d5e-4c61-a584-a1798aa5526d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:49.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:49.748162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:49.748950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:49.750198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 157/325] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:49.751614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.751913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.752194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.768392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:49.769521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:49.795954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e82e1-b5e6-446c-84ef-8685ee3437c6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:49.824967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:49.830857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:49.831517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.831799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.832076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] PATH_INFO: `/v3/groups` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.832759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:49.834768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:49.835689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 158/326] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:49.836523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:49.862045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 169/327] 10.222.0.36 () {60 vars in 1134 bytes} [Mon May 23 10:37:49 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:37:49.863073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:49.899441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:49.959890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3b4e1-df07-4b18-b799-5b0fee893ce3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:49.979711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70063-9b34-4589-b134-982cb4af910d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:49.997824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:49.998634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:49.998925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:49.999202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:49.999987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:50.000300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.001759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.010919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 159/328] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:50.017871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:50.020731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.021480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.022322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.022601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.034008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:50.042357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:50.043279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:50.047344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 170/329] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:50.049343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:50.109045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:50.153330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c42c-c56d-4ca7-8f29-90c448656f96 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:50.167486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:50.179310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac58b1-3bcf-4728-98ff-7f4d93dc6154 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:50.198516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.199749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.200236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.200702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] PATH_INFO: `/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.202173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd69cfa2d9044fc90dea5e0ce3235bf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.203811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.205647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 160/330] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:50 2022] GET /identity/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:50.211930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:50.222395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.223764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.224281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.224811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.226308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:50.226945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.228706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.240802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 171/331] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:50.247565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:50.335025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa0e6-5bf9-493d-9737-39996c5ea8f1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:50.382898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda09c0-a5f5-4892-9a8a-d8ae0e9f02f1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:50.383229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.383997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.384285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.384566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] PATH_INFO: `/v3/system/users/2cd69cfa2d9044fc90dea5e0ce3235bf/roles/4262ac95da0148d29d3d912e069159dc` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.387023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2cd69cfa2d9044fc90dea5e0ce3235bf, role_id=4262ac95da0148d29d3d912e069159dc)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.388439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.399781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 161/332] 10.222.0.36 () {58 vars in 1334 bytes} [Mon May 23 10:37:50 2022] PUT /identity/v3/system/users/2cd69cfa2d9044fc90dea5e0ce3235bf/roles/4262ac95da0148d29d3d912e069159dc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:37:50.406653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:50.431799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.432718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.433009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.433286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] PATH_INFO: `/v3/users/9974ab349b674f6cb650a99abb758d1c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.434700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] RBAC: Authorizing `identity:get_user(user_id=9974ab349b674f6cb650a99abb758d1c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.437104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f811b4-593b-4783-8f57-6aedc977b743 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.439127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 172/333] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:50 2022] GET /identity/v3/users/9974ab349b674f6cb650a99abb758d1c => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:50.445435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:50.506650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:50.574829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ed079-f6fc-4c76-82bc-308dd78659d5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:50.618607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:50.619585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:50.620053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:50.620504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:50.621634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] RBAC: Adding query filter params (role.id=63df59ac99514b3983611a6b633ec83f, user.id=9974ab349b674f6cb650a99abb758d1c, scope.system=all) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:50.622155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:50.623855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded07fc9-cd93-43ea-b482-93a615f9c62d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:50.637467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 162/334] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 23 10:37:50 2022] GET /identity/v3/role_assignments?user.id=9974ab349b674f6cb650a99abb758d1c&scope.system=all&role.id=63df59ac99514b3983611a6b633ec83f => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:52.762332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476daec1-0580-4d08-99bb-c92f0f3d9c2e None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:52.762841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476daec1-0580-4d08-99bb-c92f0f3d9c2e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:52.762917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476daec1-0580-4d08-99bb-c92f0f3d9c2e None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:52.764406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 173/335] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:52.770342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:52.772151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:52.772151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:52.792377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:52.793582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:52.874892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 163/336] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:52.911682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476daec1-0580-4d08-99bb-c92f0f3d9c2e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.046963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476daec1-0580-4d08-99bb-c92f0f3d9c2e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:53.097036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:53.098240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.098745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.099253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.114706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-45a187a4-1fcf-4e63-b526-f652176d693c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:53.123371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:53.124327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:53.125135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:53.132546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 174/337] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:53.138041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.155901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7ed5-0854-4858-99a1-a88c5d38e6ce None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.156610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7ed5-0854-4858-99a1-a88c5d38e6ce None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.156949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe7ed5-0854-4858-99a1-a88c5d38e6ce None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.158863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 175/338] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:53.165058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231972-9760-4666-bf63-bf7626215ade None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.165350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231972-9760-4666-bf63-bf7626215ade None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.165638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13231972-9760-4666-bf63-bf7626215ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.192028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-13231972-9760-4666-bf63-bf7626215ade None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:53.193241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-13231972-9760-4666-bf63-bf7626215ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:53.198338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.252851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.289581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc72584-a2f7-470c-a9af-629fa027f31b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:53.332508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:53.333548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.334097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.334773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.335715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:53.336246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:53.337819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f867b1e-6492-40b2-8106-465affad667d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:53.347489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 164/339] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:53.349062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fb5d-72ba-4ddc-8a76-470473b8c379 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.353502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fb5d-72ba-4ddc-8a76-470473b8c379 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.354116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fb5d-72ba-4ddc-8a76-470473b8c379 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.355278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 176/340] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 10:37:53.355894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 165/341] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:53.358282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13231972-9760-4666-bf63-bf7626215ade None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.362094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.362590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.363105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.400968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:53.402815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:53.548968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 166/342] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:53.550237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.568315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13231972-9760-4666-bf63-bf7626215ade None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:53.602416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.634676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:53.635344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.635630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.635907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.704698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.736817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d513a-7574-40d8-829b-9cea7de1a886 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:53.752263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:53.754135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147532f-e5ba-43cf-94b8-2dc01ccc9de1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:53.777134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:53.778017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.778306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.778579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.789679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:53.797703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:53.798661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:53.799450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 167/343] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:53.801744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.868203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 177/344] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:53.870557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383278e-e617-4df2-a562-c4fd99416e1a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.871066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383278e-e617-4df2-a562-c4fd99416e1a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.871541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383278e-e617-4df2-a562-c4fd99416e1a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.877352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 178/345] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:53.878511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383278e-e617-4df2-a562-c4fd99416e1a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:53.891494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.911843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a683c1-6067-4e76-b4eb-a900636cf83b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:53.946794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:53.947592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.948083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.948196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.956041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:53.960148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-11c867bc-b9b0-40f5-b354-ea78f1ef84b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:53.969601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:53.970604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 168/346] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:53.971460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:53.972182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:53.972182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:53.997735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:53.998974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:54.001991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383278e-e617-4df2-a562-c4fd99416e1a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.046335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.047148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.047440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.047723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.048505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:54.048819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.050444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.060946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 179/347] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:54.061824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.082748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 169/348] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:54.083843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.188828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.216645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f91bc-6427-4626-aee0-b5f3011a314b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.230689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073c4e8-50f9-44ee-b07b-7fcf26c43b87 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.257119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.265852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.266755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.267043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.267330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.268541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.273372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.273372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 180/349] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:54.273372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.281832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.282555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.282841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.283115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.283865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:54.284170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.285566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.295656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 170/350] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:54.295850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.392027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7428e-cbde-4513-8c25-f48518ddaeef None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.428347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a553dd5-d65a-4296-a8ae-ab10b6947961 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.434437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.434619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.434870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.435145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.472795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.473534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.473817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.475204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.477389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.486426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-eadfd973-905d-4809-9879-966118eb6b1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:54.495170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:54.496072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:54.496899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 171/351] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:54.498178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c9a6d-7a4e-4c1c-b8d8-fa8e0ad7f87c None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.498178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c9a6d-7a4e-4c1c-b8d8-fa8e0ad7f87c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.498351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c9a6d-7a4e-4c1c-b8d8-fa8e0ad7f87c None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.500605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 172/352] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:54.500605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c9a6d-7a4e-4c1c-b8d8-fa8e0ad7f87c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.516235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.520250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.522527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.629429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c9a6d-7a4e-4c1c-b8d8-fa8e0ad7f87c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.671550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 181/353] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:54.672491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.673434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.674791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.674791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.675002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.692752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.694768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.696838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 173/354] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:54.697772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:54.766191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.802527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:54.808962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23b4f-92a2-4c0d-ae4b-ac2fef6474d1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.814992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a937d7e-a901-4e53-a374-e2abe820ed25 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:54.849115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.850633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.851100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.851376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.854792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:54.855465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.856407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.856407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.857041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:54.857155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.858575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab24d-12a5-49b2-a5f7-922787f8b15f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.868544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 174/355] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:54.869814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:54.870144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:54.870426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:54.889573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:54.891631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:54.942608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:54.944122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:54.973911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 175/356] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:54.975200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.038097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.062203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.070116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 182/357] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:55.071003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.100666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39a35d-b1a5-4407-8feb-577d7d60e2c7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.136458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.137327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.137616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.137895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.139151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.140874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bed4bd-0ea1-4598-9b58-826faa2eaa3a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.142779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 176/358] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:54 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.143824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452aae2-5411-4142-8d4d-8965ff98d391 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.144165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452aae2-5411-4142-8d4d-8965ff98d391 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.144445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452aae2-5411-4142-8d4d-8965ff98d391 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.145681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 177/359] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:55.146996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452aae2-5411-4142-8d4d-8965ff98d391 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.184655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4565e-debb-4433-92bc-8bcd2387a0d5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.226532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.227517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.227805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.228087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.228930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:55.229245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.233002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.237018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 183/360] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=63df59ac99514b3983611a6b633ec83f => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.237532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.277694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452aae2-5411-4142-8d4d-8965ff98d391 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.303150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.338712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.340064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.341472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.341585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.341859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] PATH_INFO: `/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.343343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=2cd69cfa2d9044fc90dea5e0ce3235bf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.345171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.350415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 178/361] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/users/2cd69cfa2d9044fc90dea5e0ce3235bf => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.350415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.392031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21f8d8-8fac-4564-ad51-f3365c793d85 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.446267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.447252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.447545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.447825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.461014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:55.472354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:55.473267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:55.474083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:55.475473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 184/362] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:55.475473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.487868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aade613-9efc-4b11-8550-d02a4e155dd9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.534034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.534955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.535635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.535919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.538193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.539666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7aa53-20ab-43b9-b1ce-787ab433d96b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.541444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 179/363] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.542744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.543036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.543312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.564366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:55.565439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:55.619745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574159-14da-4888-b55a-0c7d3cf0224a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.628374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.653656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.654992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 180/364] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:55.655665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.672693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.673756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.673872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.674876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.675494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.676947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.678976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 185/365] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.679731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.785238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e92e0-1a7b-4983-9c59-13c5a0b8cbaa None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.808245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a1547-cc20-427f-b743-f4d9070436af None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:55.835727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.836967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.837251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.837530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.838405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] RBAC: Adding query filter params (role.id=4262ac95da0148d29d3d912e069159dc, user.id=2cd69cfa2d9044fc90dea5e0ce3235bf, scope.system=all) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:55.838721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.840203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.852967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:55.853912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:55.854438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:55.854725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 181/366] 10.222.0.36 () {56 vars in 1308 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/role_assignments?user.id=2cd69cfa2d9044fc90dea5e0ce3235bf&scope.system=all&role.id=4262ac95da0148d29d3d912e069159dc => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.855063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:55.855628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.856157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:55.856646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:55.858484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:55.872037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 186/367] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:55.874025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:55.916113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:55.957697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.007236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6b15b-a0df-453e-97f7-5a24eed76697 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.018152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04bef7-ffcd-46ca-b4f0-8a8bc35c203a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.060662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.062134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.062588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.062870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.064172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.065671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.066590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.067508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.067709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 182/368] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:56.068036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.069017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.069160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.082247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:56.090942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:56.092001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:56.092816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 187/369] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:56.093729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.209160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e0e5b-b324-499f-8fb7-bd3c07dd4604 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.221404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.226893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2308a-7324-408e-b7ce-23f7326173e9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.250408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.258433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.259436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.260409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.260914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.262311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:56.262995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.265548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.273776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 183/370] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:56.276184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.280265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.281284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.281893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.282282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.395197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.398055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.409426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3cdb5-e51f-4825-9434-ce2f25757487 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.452481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.453307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.454484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.454484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.455025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=12cb24e441a941148f32f0504181b73b) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:56.455443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.457012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.465004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 184/371] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=63df59ac99514b3983611a6b633ec83f => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:56.466081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.518234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.550467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 188/372] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:37:56.558714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.586362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.599515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e25175-a92b-4443-b421-403b0e532ebd None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.651768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.652384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.652660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.652944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.653710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:56.654054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.655479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.670249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 185/373] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:56.675955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.678857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036afdb-fe95-4427-a0de-2b90734842d0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.715472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.716356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.716638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.716914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.719072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.720547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.722323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 189/374] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:56.728727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:56.814184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.817308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01d3ab-6fa4-4744-99a9-990d4af98a8a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.856312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.857217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.857422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:56.857687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.858352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.893885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff04910-45b7-4ab7-a3b5-680e22580b8c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:56.934568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.936102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.982667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:56.983603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:56.983897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:56.984181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:56.985417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:56.986914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:56.988979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 190/375] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:56.998873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:57.050245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:57.055827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 186/376] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:37:57.062906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:57.180164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ba4cc-b31b-436b-a528-0e0b7bcf0d60 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:57.222250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:57.222978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.223398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.223678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.224793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=12cb24e441a941148f32f0504181b73b) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:57.225111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:57.226989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:57.233549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 191/377] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:37:56 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:57.269078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed54689-19b5-4224-9d29-02d1180213ec None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:57.324632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:57.325800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.326284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.326728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] PATH_INFO: `/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.328184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] RBAC: Authorizing `identity:get_project(project_id=a6dd918cb8494a52af5c7b9bb5c60e89)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:57.329829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:57.331745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 187/378] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:57 2022] GET /identity/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:57.337431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:57.363871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:57.491938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee943d-b34d-417f-ba89-3e9453ac6e2a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:57.539255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:57.540139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.540427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.540701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] PATH_INFO: `/v3/groups/1b4c04934eac4e7e85af5db928a4febd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.553298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c04934eac4e7e85af5db928a4febd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:57.554802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226194-caaf-4866-98d0-d290e590e8ab None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:57.556863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 192/379] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:57 2022] GET /identity/v3/groups/1b4c04934eac4e7e85af5db928a4febd => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:57.562466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:57.675356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:57.721186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b2b-2c43-4908-b85c-89e4810000ad None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.721519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b2b-2c43-4908-b85c-89e4810000ad None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.721749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602b2b-2c43-4908-b85c-89e4810000ad None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.723291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 193/380] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:37:57.723871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2d38-7b6e-40f7-ad25-fa1b478cfe2e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:57.728673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.728964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.729241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.746765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:57.747869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:57.796276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:57.796943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:57.797232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:57.797510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:57.798376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] RBAC: Adding query filter params (group.id=1b4c04934eac4e7e85af5db928a4febd, role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=a6dd918cb8494a52af5c7b9bb5c60e89) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:57.798689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:57.800140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:57.816176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 188/381] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:37:57 2022] GET /identity/v3/role_assignments?group.id=1b4c04934eac4e7e85af5db928a4febd&scope.project.id=a6dd918cb8494a52af5c7b9bb5c60e89&role.id=081e04042be64de0ac1c857efdae3324 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:57.825534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 194/382] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:57.872043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:57.998029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e310f9c-9e98-45d9-82d4-81404e5dbbcb None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:58.034338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:58.041539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:58.042397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:58.042690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:58.043531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:58.059801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:58.067859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:58.068864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:58.069665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 189/383] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:58.079125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:58.094865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:58.220613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96433c-8459-4f37-9f20-6d882ac1d18b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:58.255792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:58.256399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:58.256683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:58.258208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:58.259233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:58.259770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:58.261462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:58.280180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 195/384] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:58.287608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:58.413578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc08c2-aaf2-4f92-bde7-6ff0a54576e7 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:58.453412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:58.454302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:58.454590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:58.454872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:58.458635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:58.460679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:58.462516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 190/385] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:58 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:58.467877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:58.566419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:58.591926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226be5b-0f7b-4494-8fce-813d58beeb1d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:58.625547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:58.626435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:58.626715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:58.626987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:58.628176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:58.629632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c10b7d-8809-499f-a624-c6db121bc287 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:58.636576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 196/386] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:58 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:58.637971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:58.724287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:58.786092 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934b3be-84a3-4aa1-8185-445ff8342c1b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:58.829840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:58.830702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:58.830986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:58.831261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/groups/a587af52146c498094bcb65477bf82cb/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:58.833274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, group_id=a587af52146c498094bcb65477bf82cb, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:58.834838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4e147-4949-4e32-9c2d-209bad177c42 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:58.863005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 191/387] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 23 10:37:58 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/groups/a587af52146c498094bcb65477bf82cb/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:37:59.096792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68acd5-a109-4aac-b24f-a73552f084e2 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.097302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68acd5-a109-4aac-b24f-a73552f084e2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.097302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68acd5-a109-4aac-b24f-a73552f084e2 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.098996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 197/388] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:59.104971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f142e6-b2ae-45bc-99ee-92a9be186380 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.105502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f142e6-b2ae-45bc-99ee-92a9be186380 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.105502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f142e6-b2ae-45bc-99ee-92a9be186380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.124658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-20f142e6-b2ae-45bc-99ee-92a9be186380 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:59.125893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-20f142e6-b2ae-45bc-99ee-92a9be186380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:59.200130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 192/389] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:37:59.221537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68acd5-a109-4aac-b24f-a73552f084e2 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:59.305755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7581c-ef3f-4260-802b-4ac00cb323ad None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.306455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7581c-ef3f-4260-802b-4ac00cb323ad None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.306951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7581c-ef3f-4260-802b-4ac00cb323ad None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.308525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 193/390] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:37:59.313943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.314512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.315006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.334815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68acd5-a109-4aac-b24f-a73552f084e2 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:59.347044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:37:59.348539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:37:59.374322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:59.374322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.374322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.374322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.378403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:59.382049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:59.390489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:59.391460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:37:59.392263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 198/391] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:37:59.399420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:59.452265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:59.489546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 194/392] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:37:59.520622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:59.590809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aaae6-e19a-4314-a189-66a188d041cb None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:59.678749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:59.679463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.679824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.680174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.680993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:37:59.681469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:59.683042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:59.692242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db2e3e9-5251-463f-9f5b-ee121dd10d5c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:59.700880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 199/393] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:37:59.707899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:59.766822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:59.767649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.767941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.768224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.791236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:37:59.802113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:37:59.803231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:37:59.804201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:37:59.805927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:59.809985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 195/394] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:37:59.813665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:37:59.877219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a2861-6f9b-42fe-942f-cf61b93c37b5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:37:59.892173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:37:59.940637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:37:59.941791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:37:59.942556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:37:59.943020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:37:59.947367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:37:59.949357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:37:59.959999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 200/395] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:37:59.965889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.017486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c3e67-42f8-40c7-a0af-914bf335cb8c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.077817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.078768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.080969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.081397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.082627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] RBAC: Adding query filter params (name=member) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:00.083477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.085609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.104233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 196/396] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:00.107666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.148080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa7a9a-9ca5-49a4-8f56-7ad5eb32e3a6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.194573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.211060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.212673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.212971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.213828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.215403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.217021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98074b3-67f2-43cc-9e05-84fa07024c33 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.219185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 201/397] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:37:59 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:00.224348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6b5-9b9e-4621-a647-d7f114579250 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.224348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6b5-9b9e-4621-a647-d7f114579250 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.224348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6b5-9b9e-4621-a647-d7f114579250 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.226123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 202/398] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:00.227067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0d6b5-9b9e-4621-a647-d7f114579250 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.285737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb37df-d8a3-4263-901e-2d79a742f13f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.288587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.333408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.334972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.335564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.336099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.337687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.340034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2842a-8e06-4184-bc73-1b29e442d632 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.342793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 197/399] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:00.344120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1af2-d956-48ba-8fcd-edaa62e08fdc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.345128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1af2-d956-48ba-8fcd-edaa62e08fdc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.345574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d1af2-d956-48ba-8fcd-edaa62e08fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.358333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0d6b5-9b9e-4621-a647-d7f114579250 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.367630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-160d1af2-d956-48ba-8fcd-edaa62e08fdc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:00.369533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-160d1af2-d956-48ba-8fcd-edaa62e08fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:00.401104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.402367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.402721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.403065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/groups/a587af52146c498094bcb65477bf82cb/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.406141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee569fa6f64d5a800246a070eb1d3d, group_id=a587af52146c498094bcb65477bf82cb, role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.408288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.431075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 203/400] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 23 10:38:00 2022] PUT /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d/groups/a587af52146c498094bcb65477bf82cb/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:00.432455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.454781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 198/401] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:00.463267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596288-3ca5-4bbe-8717-75087efbe884 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.463570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596288-3ca5-4bbe-8717-75087efbe884 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.463847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e596288-3ca5-4bbe-8717-75087efbe884 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.466754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 199/402] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:00.475372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.475693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.475936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.500184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:00.501426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:00.529239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.581462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc9705-ff97-4983-9b03-8371d67379ca None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.586373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 200/403] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:00.587206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.620679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.622062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.622554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.623062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.624683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.626466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.628418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 204/404] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:00.629489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.639437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.736386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d86f0-45ef-47b8-b90f-a9b73fba2571 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.740748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8555b70-cb2c-4983-90ca-74fd1949a949 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.766503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.779803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.781119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.781608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.782134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.782239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.782936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.783228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.783531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.796486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:00.797924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:00.805156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:00.806155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:00.806952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:00.810097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:00.810960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:00.811767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:00.812379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 205/405] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:00.812379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.814555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 201/406] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:00.815751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:00.916043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:00.928354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f5ba0-6404-4642-a5e4-9f47485419b6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.953457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a186-7e60-4c22-acbc-ad9e00fe565c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:00.964941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:00.965974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:00.966470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:00.966950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:00.967943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:00.968475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:00.970146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:00.980468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 202/407] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:00.981600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.000597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.001991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.002640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.003119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/groups/a587af52146c498094bcb65477bf82cb/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.005449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, group_id=a587af52146c498094bcb65477bf82cb, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.007251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.027644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 206/408] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 23 10:38:00 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/groups/a587af52146c498094bcb65477bf82cb/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:01.028944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.141358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.141893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339dbea-a232-4bf4-85dd-6fef8bfa4529 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.202080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.203185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.203750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.204248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.205226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:01.205739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.207369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.216861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3108be4-a5b4-44f6-b855-538054fa11a9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.221312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 203/409] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:01.227470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.258070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.267426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.268817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.269330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.269837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] PATH_INFO: `/v3/groups/1b4c04934eac4e7e85af5db928a4febd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.274717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c04934eac4e7e85af5db928a4febd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.276640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.282089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 207/410] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:01 2022] GET /identity/v3/groups/1b4c04934eac4e7e85af5db928a4febd => generated 229 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:01.287429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.389436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034361-32fe-4fd6-b815-2e280ada88d0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.415681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81441cb-00b7-45a9-a9a8-ad1f4407e497 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.427997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.428855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.429141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.429422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.430733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.432200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.434175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 204/411] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:01 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:01.434822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.440301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.469464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.470620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.470620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.470893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] PATH_INFO: `/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.472143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] RBAC: Authorizing `identity:get_project(project_id=a6dd918cb8494a52af5c7b9bb5c60e89)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.473538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b84e6-11e2-434d-b36d-d045a330468b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.475676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 208/412] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:01 2022] GET /identity/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:01.476923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83480c-c533-492e-8c78-5a20ac3d801d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.477213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83480c-c533-492e-8c78-5a20ac3d801d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.477582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83480c-c533-492e-8c78-5a20ac3d801d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.478973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 209/413] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:01.483574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83480c-c533-492e-8c78-5a20ac3d801d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.539159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.558697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfede-7dae-4a30-b7df-029ce82b6205 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.593944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.594726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83480c-c533-492e-8c78-5a20ac3d801d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.595409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.595923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.596413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.597837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.599552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797822b-60ed-4783-a95e-0068a5827d6f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.601837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 205/414] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:01 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:01.603324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.603810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.604293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.630245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:01.632651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:01.641568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.643215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.643718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.644194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] PATH_INFO: `/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89/groups/1b4c04934eac4e7e85af5db928a4febd/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.647902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6dd918cb8494a52af5c7b9bb5c60e89, group_id=1b4c04934eac4e7e85af5db928a4febd, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.649744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.669556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 210/415] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 23 10:38:01 2022] PUT /identity/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89/groups/1b4c04934eac4e7e85af5db928a4febd/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:01.673994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.742719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 206/416] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:01.770573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.771186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:01.900912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19910ec-c1b2-4494-b333-463ae7f1a7d0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.921999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:01.950291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b996ac-7018-43ce-ad2f-91da7385ccba None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:01.953015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.954819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.955050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.955331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b/groups/a587af52146c498094bcb65477bf82cb/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:01.963414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] RBAC: Authorizing `identity:create_grant(project_id=12cb24e441a941148f32f0504181b73b, group_id=a587af52146c498094bcb65477bf82cb, role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:01.965217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:01.990887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 211/417] 10.222.0.36 () {58 vars in 1442 bytes} [Mon May 23 10:38:01 2022] PUT /identity/v3/projects/12cb24e441a941148f32f0504181b73b/groups/a587af52146c498094bcb65477bf82cb/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:01.995731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:01.996864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:01.997333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:01.997893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.011565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:02.020034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:02.020991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:38:02.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:02.024615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 207/418] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:02.030703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:02.141809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:02.145898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d14975-1a10-466b-92c0-1210c4c5f47f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:02.200818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:02.201757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:02.202219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:02.202666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.203651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:02.204133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:02.205722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:02.218713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 212/419] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:38:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:02.226368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:02.354256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40e56a-2714-4b02-9de8-f2c2820046b2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:02.392231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:02.392889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:02.393179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:02.393518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.405249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:02.483816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:02.485279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafadd92-d6e2-4c26-9a4a-21ac35143e95 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:02.614394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 208/420] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:38:02.620255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:02.778820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73f06c2-ee0f-4ae1-86f6-2258af8e50d5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:02.811722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fe90-c82f-417e-a837-e5341fb43135 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:02.812027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fe90-c82f-417e-a837-e5341fb43135 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:02.812277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5fe90-c82f-417e-a837-e5341fb43135 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.815077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 209/421] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:02.819005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:02.819290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:02.819568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.836337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:02.837530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:02.850227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:02.851115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:02.851400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:02.851676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:02.853758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:02.855261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:02.857536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 213/422] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:02 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:02.863549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:02.890463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:02.896948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:02.972139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ab50c-dde4-40af-b5b0-20ad4d92d975 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:02.976786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 210/423] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:03.008122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:03.008985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.009269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.009545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.010761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:03.012202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:03.014143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 214/424] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:02 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:03.015997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:03.021994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:03.189053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec01167-6ef1-4a14-8515-f9e5975d7618 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:03.203362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:03.210784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237d523-1ef8-4f65-a00c-905c5bedfa41 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:03.247196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:03.248127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.248418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.248698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.258832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:03.259494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.259783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.260064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.260902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:03.261218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:03.263729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:03.272506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:03.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:03.281494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:03.282324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:03.284795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 215/425] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=63df59ac99514b3983611a6b633ec83f => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:03.285042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 211/426] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:03.287806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:03.445669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855f5e5-bee7-43b3-b8ef-8335a758d099 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:03.496191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:03.496873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.497156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.497432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.498209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:03.498515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:03.499894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df106d-3e52-4c86-b7b9-c2ca35aa6310 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:03.514059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 216/427] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:03.516460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:03.548410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:03.625606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bbf94-1d68-42de-8560-c19bb1b5ca6b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:03.669701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:03.670791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.671276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.671794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.749688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:03.751537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:03.785049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:03.845047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8577d-77f9-4b27-b3d5-32b745c1ba62 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.845691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8577d-77f9-4b27-b3d5-32b745c1ba62 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.846082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8577d-77f9-4b27-b3d5-32b745c1ba62 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.848696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 217/428] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:03.853723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:03.854129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:03.854606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:03.863197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 212/429] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:03.876107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:03.879806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:03.880588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:03.908042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:03.970629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 218/430] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:03.996859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.016588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9adadf2-9648-4316-87a5-3ca87a1ad081 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.048330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.056013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.056878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.057273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.057563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] PATH_INFO: `/v3/projects/08ee569fa6f64d5a800246a070eb1d3d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.058915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] RBAC: Authorizing `identity:get_project(project_id=08ee569fa6f64d5a800246a070eb1d3d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.060391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 213/431] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/projects/08ee569fa6f64d5a800246a070eb1d3d => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:04.069405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.131418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb1ec-1413-4867-82ae-5e0f618c24c4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.187895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.189249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.189675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.190204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.204259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:04.212298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e47bc-1b38-43d0-8f7a-bd66fcab0c94 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:04.214618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:04.215476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 10:38:04.216244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:04.217237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 219/432] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:04.226234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.250374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.268726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.270891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.271929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.272354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.274718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.276203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.278069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 214/433] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:04.284473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.361134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8d774-f95f-470b-8f7f-583cd9550730 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.361911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.404474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.405298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b3fc-7c5d-4266-ab4b-7127fd3ee79e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.405770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.406416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.406902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.407855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] RBAC: Adding query filter params (name=member) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:04.408364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.410019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.420478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 220/434] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:04.426818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.443683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.444343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.444683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.444965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.445853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=08ee569fa6f64d5a800246a070eb1d3d) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:04.446202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.447665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1a747-9f75-4a6f-b895-519394f8feb5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.461981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 215/435] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=08ee569fa6f64d5a800246a070eb1d3d&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:04.522069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.549964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09629b21-30f5-491b-a63d-b3ca2352053f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.580557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc3b5-2d55-4c77-b14a-270d68fb2728 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.580855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc3b5-2d55-4c77-b14a-270d68fb2728 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.581141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc3b5-2d55-4c77-b14a-270d68fb2728 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.582840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 216/436] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:04.587377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.588049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.588335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.588455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.588612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.588712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.588894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.608049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:04.609308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:04.678031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.680568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.686530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 217/437] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:04.712308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.772383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.797224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:04.801824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 221/438] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:04.808930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:04.853000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744bd53-8f91-424d-94b1-fbf231eec132 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.900070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.900872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.901159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.901442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.914990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:04.922052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b163904-931a-45ee-ab7c-c19336c39a7e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-20f4ff85-336b-4563-a2d9-43013170168e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:04.922773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:04.923792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:04.924785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 218/439] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:04.925604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd592-60de-4973-8303-ad3187a9dd63 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.925891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd592-60de-4973-8303-ad3187a9dd63 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.926261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fd592-60de-4973-8303-ad3187a9dd63 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.927704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 219/440] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:04.932789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.933076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.933355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.950938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:04.952118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:04.962268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:04.963129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:04.963413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:04.963689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:04.964873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:04.966388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:04.968452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 222/441] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:04.969984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.013844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.020541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 220/442] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:05.021894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.069719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894fde-32ab-4cf1-8e19-291350d4a3ae None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.070610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.103240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.103907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.104190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.104465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.105206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:05.105512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.106923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.114856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 223/443] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:05.115631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.120299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62345275-ebc2-4ad9-bb6a-f92f7854e0ce None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.153589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.154541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.154827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.155109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.156289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.157751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.159443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 221/444] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:05.160203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.218180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b67e63-b43c-4ade-bb2b-3734bc9e1a3a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.224764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.255119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.256427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.256713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.256990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.261431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a315d-c03a-4b4c-9a6a-739bce92af5d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.267230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:05.275950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:05.276914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:05.277730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:05.278372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 224/445] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:05.279133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.279375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.297247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.297896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.298228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.298508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.365552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.367267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.375710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d1709-3529-4ea9-8cff-18c4421c8ff7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.409677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.410354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.410643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.410920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.411780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=63df59ac99514b3983611a6b633ec83f, scope.project.id=12cb24e441a941148f32f0504181b73b) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:05.412096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.413536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.419268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 225/446] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=63df59ac99514b3983611a6b633ec83f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:05.420046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.457603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.484924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 222/447] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:05.491401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.511835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.543880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7e9b7-05c6-42bb-9495-ed9ae917e33e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.583053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.583876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.584164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.584570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.585336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:05.585649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.587098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.596694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 226/448] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:05.599889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99446c4-792d-4ed3-806e-d9707adcb5e4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.602093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.641047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.641832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.642163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.645155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] PATH_INFO: `/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.645886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] RBAC: Authorizing `identity:get_project(project_id=a6dd918cb8494a52af5c7b9bb5c60e89)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.647435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.649181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 223/449] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/projects/a6dd918cb8494a52af5c7b9bb5c60e89 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:05.654578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.711671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.722516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeae3c-1909-40ea-86c3-3fee677c5ec6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.756992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.765582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.766272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.766556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.766831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.773979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfd188-21aa-42b5-9e80-a5eb211cbdce None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.813157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.814049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.814338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.814618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] PATH_INFO: `/v3/groups/1b4c04934eac4e7e85af5db928a4febd` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.815830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c04934eac4e7e85af5db928a4febd)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.817568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.819184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 224/450] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/groups/1b4c04934eac4e7e85af5db928a4febd => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:05.824690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.844457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.845917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.922040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bbe4c-936e-464a-a86c-7c823930d620 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:05.934063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:05.943691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 227/451] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:38:05.949135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:05.956056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:05.956702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:05.956980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:05.957260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:05.958113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] RBAC: Adding query filter params (group.id=1b4c04934eac4e7e85af5db928a4febd, role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=a6dd918cb8494a52af5c7b9bb5c60e89) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:05.958429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:05.959875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:05.973143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 225/452] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/role_assignments?group.id=1b4c04934eac4e7e85af5db928a4febd&scope.project.id=a6dd918cb8494a52af5c7b9bb5c60e89&role.id=081e04042be64de0ac1c857efdae3324 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:06.060343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda89bf-d78b-4ee5-b006-b762f36d6ac1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:06.095411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:06.096289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:06.096573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:06.096848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] PATH_INFO: `/v3/projects/12cb24e441a941148f32f0504181b73b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:06.098124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] RBAC: Authorizing `identity:get_project(project_id=12cb24e441a941148f32f0504181b73b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:06.100156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:06.101931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 228/453] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:38:05 2022] GET /identity/v3/projects/12cb24e441a941148f32f0504181b73b => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:06.107493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:06.127982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:06.209889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bca4a1-ee6c-44d0-8b20-f81fee362d57 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:06.244101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:06.244955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:06.245242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:06.245521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] PATH_INFO: `/v3/groups/a587af52146c498094bcb65477bf82cb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:06.246741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] RBAC: Authorizing `identity:get_group(group_id=a587af52146c498094bcb65477bf82cb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:06.248191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a85663-1748-4aa4-a2e0-323c1f56d11c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:06.249839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 226/454] 10.222.0.36 () {56 vars in 1180 bytes} [Mon May 23 10:38:06 2022] GET /identity/v3/groups/a587af52146c498094bcb65477bf82cb => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:06.256583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:06.313936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:06.352925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08200f-7efe-4121-9ef9-e6bdf9cbf2e1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:06.385753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:06.386425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:06.386708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:06.386984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:06.387808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] RBAC: Adding query filter params (group.id=a587af52146c498094bcb65477bf82cb, role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=12cb24e441a941148f32f0504181b73b) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:06.388126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:06.389556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d6ca6-6566-4f5e-b1c6-a2ae79273235 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:06.395511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 229/455] 10.222.0.36 () {56 vars in 1376 bytes} [Mon May 23 10:38:06 2022] GET /identity/v3/role_assignments?group.id=a587af52146c498094bcb65477bf82cb&scope.project.id=12cb24e441a941148f32f0504181b73b&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:09.259228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c339-7a01-488a-b7e0-11939f440580 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.259789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c339-7a01-488a-b7e0-11939f440580 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.260040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c339-7a01-488a-b7e0-11939f440580 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.261929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 227/456] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:09.268488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0a589-8bb9-425f-bea8-f8f5af8b596b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.269349 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0a589-8bb9-425f-bea8-f8f5af8b596b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.269882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0a589-8bb9-425f-bea8-f8f5af8b596b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.295238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-75d0a589-8bb9-425f-bea8-f8f5af8b596b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:09.296751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-75d0a589-8bb9-425f-bea8-f8f5af8b596b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:09.390416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 230/457] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:09.427190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c339-7a01-488a-b7e0-11939f440580 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:09.563090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:09.584491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c339-7a01-488a-b7e0-11939f440580 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:09.595557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6290f1-8239-4f27-a9d6-294033d9250a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.596145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6290f1-8239-4f27-a9d6-294033d9250a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.596624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6290f1-8239-4f27-a9d6-294033d9250a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.598483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 231/458] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:09.607319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.607880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.608345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.630246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:09.631125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.631418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.631519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:09.631695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.632734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:09.647788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:09.649987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-b5446cad-ba30-40e3-b950-62598dd706c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:09.674274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:09.675221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:09.676079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:09.676079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:09.676079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:09.676079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:09.676737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 228/459] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:09.677938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2eb8-3764-4049-9a4f-419048b27496 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.678247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2eb8-3764-4049-9a4f-419048b27496 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.678527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b2eb8-3764-4049-9a4f-419048b27496 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.684460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 229/460] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:09.685909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.686228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.686507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.695269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:09.712354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:09.713562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:09.716262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 232/461] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:09.717317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:09.803516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 230/462] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:09.804791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:09.861652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10856c0f-f219-4b00-9793-708e1775c96c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:09.879728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:09.898390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:09.899063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.899346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.899624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.900344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:09.900652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:09.902061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27c036-cdeb-4d22-a2cb-0d19539d578f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:09.913707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 233/463] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:09.918951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6c609-b741-480c-9157-e7599bcfbcaa None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:09.919718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5048ca-7f03-4c9d-b684-2fb0a3ff2f28 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.920184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5048ca-7f03-4c9d-b684-2fb0a3ff2f28 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.920609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5048ca-7f03-4c9d-b684-2fb0a3ff2f28 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.922081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 234/464] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:09.922918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5048ca-7f03-4c9d-b684-2fb0a3ff2f28 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:09.956414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:09.957202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:09.957489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:09.957765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:09.968860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:09.970366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:09.975701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:09.991142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:09.992024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:09.992857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:09.992857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:09.992857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:09.992857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:09.993467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 231/465] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:09.994108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.038184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5048ca-7f03-4c9d-b684-2fb0a3ff2f28 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.073227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.074280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.074732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.075167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.086505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.088149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.101982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12875f43-b0cc-4cd3-af98-3689b3c74944 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-4740529f-63c2-4375-8dc5-111a0394e695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:10.105920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:10.106958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:10.108006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:10.108006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:10.108006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:10.108006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:10.109546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 235/466] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:10.110920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.111348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.111824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.129298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.130957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:10.132336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:10.147058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.147733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.148013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.148293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.149006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.150491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.174017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 232/467] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:09 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:10.174017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.246536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 236/468] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:10.247850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.296263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.338941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea074-9909-404b-adb6-e841f4cbaa7c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.384533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.385435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.385730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.386047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.387512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:10.387512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.388512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.394013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324df8d2-a2d7-42e2-8bc9-91583b01f403 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.406752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 233/469] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:10.407595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.434647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.435319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.435609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.435889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.436783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:10.437093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.438553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.447624 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.453707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 237/470] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:10.454839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.528051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df609c2-7f98-47c9-a7cf-71e616d14a6a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.546053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.563071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbb36d-a35d-4041-b2ae-4c832ad48c4d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.568210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.569138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.569427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.569702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.581082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.582606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.601576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.602278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.602565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:10.602746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:10.603586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:10.604455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:10.604455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:10.604455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:10.604455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:10.604663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.604663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.604789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 234/471] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:10.605413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.609551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090da398-7ac5-4fba-ba35-c2146c8f693e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.642517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 238/472] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:10.645831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b692c8-a05d-45fe-bcba-12442f59fe3f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.646505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b692c8-a05d-45fe-bcba-12442f59fe3f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.647017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b692c8-a05d-45fe-bcba-12442f59fe3f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.649125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 239/473] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:10.651105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b692c8-a05d-45fe-bcba-12442f59fe3f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.711083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.726911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe53d8-fa69-4da2-817f-a32efa7b9b1b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.770254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.771023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.771311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.771593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.772746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.773852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86506a-787b-45b7-90d4-e9c131985787 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.806295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 235/474] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:10.806488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b692c8-a05d-45fe-bcba-12442f59fe3f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:10.807823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.808461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.808748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.838360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:10.839971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:10.861074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:10.862586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:10.863044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:10.863393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:10.864200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:10.864598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:10.866811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:10.885940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 240/475] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:10.891949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:10.895060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:10.923364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 236/476] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:10.948632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:11.007726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96531-bd3e-479c-a5cb-25ea5cf6c4c6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:11.035640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:11.049728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:11.050430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:11.050713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:11.050987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:11.051769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:11.053258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:11.065195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071dbc-400d-4179-9c0f-4ccf967d82b4 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:11.077158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 241/477] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/users => generated 249 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:11.110569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:11.111418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:11.111809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:11.112192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:11.113081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:11.114807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:11.124675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.sql.core [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 10:38:11.124675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 10:38:11.124675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [parameters: {'id': '03b21c672eb6447484dc716f4ac30654', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 10:38:11.124675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91696) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application conn.query(q) May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result.read() May 23 10:38:11.143097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return ref.to_dict() May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:38:11.144038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application next(self.gen) May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application yield resource May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application next(self.gen) May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.session.rollback() May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.force_reraise() May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise self.value May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application session.commit() May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 10:38:11.144929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._prepare_impl() May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.session.flush() May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._flush(objects) May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application compat.raise_( May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application flush_context.execute() May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rec.execute(self) May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 10:38:11.146766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:38:11.147666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application conn.query(q) May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result.read() May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:38:11.148462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [parameters: {'id': '03b21c672eb6447484dc716f4ac30654', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:11.149228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 10:38:11.150132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:11.150937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 237/478] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 23 10:38:10 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:11.156354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:11.226338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:11.273205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446d18-9e48-4656-86a5-e8bc85362768 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:11.311850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:11.312672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:11.312955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:11.313230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:11.325371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-cf08601f-2deb-481a-9989-19754ce4c6a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:11.334083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:11.334988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:11.335776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 242/479] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:11.341101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:11.448042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13644b-2eab-4595-8ff7-cf27420ab045 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:11.484388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:11.485058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:11.485341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:11.485616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:11.486392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:11.486700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:11.488084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0c787-399b-4d70-a24c-533cc6b482d3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:11.493415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:11.502046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 238/480] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:12.812128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df052-9d7a-44f9-8ba5-c113dff3495d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:12.812666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df052-9d7a-44f9-8ba5-c113dff3495d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:12.812666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df052-9d7a-44f9-8ba5-c113dff3495d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:12.814284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 243/481] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:12.820449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:12.821260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:12.821781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:12.839907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:12.841392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:12.911419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 239/482] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:12.935143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df052-9d7a-44f9-8ba5-c113dff3495d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.065614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df052-9d7a-44f9-8ba5-c113dff3495d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.088716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:13.110476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:13.111273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.111552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.111826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.131427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:13.139613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:13.140765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:13.141577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:13.144070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 244/483] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:13.149964 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.322515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb9ebc-598d-4a82-9532-7e2edc1f6fcd None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.359778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:13.370815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec601738-e300-4e33-8399-28068880d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:13.371937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601738-e300-4e33-8399-28068880d898 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.372455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601738-e300-4e33-8399-28068880d898 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.372941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601738-e300-4e33-8399-28068880d898 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.373930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec601738-e300-4e33-8399-28068880d898 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:13.374483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec601738-e300-4e33-8399-28068880d898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:13.376144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec601738-e300-4e33-8399-28068880d898 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:13.386811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 240/484] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:13.392145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.505006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810fa2-e673-4911-9fef-1486a00aab52 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.545394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:13.546684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.547264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.547742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.559595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1914e-bb73-4fe9-aa58-d8ae04f6aa1d None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.559880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1914e-bb73-4fe9-aa58-d8ae04f6aa1d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.560152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1914e-bb73-4fe9-aa58-d8ae04f6aa1d None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.561517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 241/485] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:13.568132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:13.569617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:13.572940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a7915-04e6-407c-8c3e-2c02bc0a9c8f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.573176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a7915-04e6-407c-8c3e-2c02bc0a9c8f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.573449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a7915-04e6-407c-8c3e-2c02bc0a9c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:13.589817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:13.590874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:13.591899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:13.591899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:13.591899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:13.591899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:13.593707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 245/486] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:13.599523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.615641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-dc8a7915-04e6-407c-8c3e-2c02bc0a9c8f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:13.616832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dc8a7915-04e6-407c-8c3e-2c02bc0a9c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:13.646106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:13.718120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 242/487] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:13.718120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4173-b18a-4ec8-83e3-3b454767d07f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.718120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4173-b18a-4ec8-83e3-3b454767d07f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.718120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4173-b18a-4ec8-83e3-3b454767d07f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.718120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 243/488] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:13.721828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832953-9e2a-44bd-bd6d-acb9b29e699f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.735709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.735990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.736264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.756242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:13.757595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:13.771519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:13.772689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.773189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.773834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.775223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:13.775954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:13.778147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:13.794776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 246/489] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:13.795857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.858050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 244/490] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:13.858050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:13.922003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:13.927492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:13.932716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c4fbd-956c-405e-8bbf-8df9fd7e49a9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.969875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:13.971037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:13.971493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:13.971938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:13.980614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a11a7-7dc2-408a-8f41-ad28ae983f6c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:13.984123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:13.993110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:13.993997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:13.994811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:13.997264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 247/491] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:13.998296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.023083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.023871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.024264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.024540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.031741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.033198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:14.048564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:14.049500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 245/492] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:14.051071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.110284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8f569-9041-4889-8249-47626744d451 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.147449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.148345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.148628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.149062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.151719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ed35d-22b9-4bf0-bdb4-4774d656976e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.159429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:14.166147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-44d2cf6e-1211-49ed-8aac-16422a258d1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:14.167780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:14.168645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:14.169452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 248/493] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:14.170783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c5ee-3b14-483f-8c84-a018f1674df0 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.170783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c5ee-3b14-483f-8c84-a018f1674df0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.170783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881c5ee-3b14-483f-8c84-a018f1674df0 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.174072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 249/494] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:14.174072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c5ee-3b14-483f-8c84-a018f1674df0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.188077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.196980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.197646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.197927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.198230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.199347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:14.199347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.200673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.213103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 246/495] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:14.213103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.294969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881c5ee-3b14-483f-8c84-a018f1674df0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.319295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce57d0-61e4-4f66-8be7-b88e8f758603 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.329447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.330153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.330442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.330725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.331571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:14.331886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.333376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964eae5c-6cd7-49f5-9db2-c6558c95823f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.359301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.361203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.361642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.362201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.363664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:14.364219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.367654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.374846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 250/496] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:14.376288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.376592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.376872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.383386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 247/497] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:14.384243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.398570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:14.399776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:14.420772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.464031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.470750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 251/498] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:14.471663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.501498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516c96-8f73-45b9-b3fb-a11de4c84a31 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.538348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.539175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.539461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.539735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.551141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.553190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:14.570370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:14.571308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:14.572143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:14.572143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:14.572143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:14.572143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:14.572143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 248/499] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:14.572419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.581548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b9d7-c26b-4ffe-b873-163e2a439451 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.616137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.616712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.617002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.617275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.657878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.670581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0b59-9326-4946-a14c-7e8ad70420bd None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.679916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.684361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.685826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bbb06d-29b2-4800-8b77-cbba5b9f7712 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.703024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.703799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.704084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.704360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.715202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.716675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:14.734298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:14.735276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:14.736098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:14.736098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:14.736098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:14.736098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:14.736098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 249/500] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:14.736402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.783290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 252/501] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:14.784616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6344-f3cd-432a-bb86-6724e97e27d7 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.784905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6344-f3cd-432a-bb86-6724e97e27d7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.785182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e6344-f3cd-432a-bb86-6724e97e27d7 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.787324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 253/502] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:14.789254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e6344-f3cd-432a-bb86-6724e97e27d7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.834498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b1964-699e-41cc-856b-f06d61ab5d73 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.867646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.868359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.868640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.868919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.876209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.879669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:14.884792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:14.888151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:14.889183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:14.890477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 250/503] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:14.891211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e6344-f3cd-432a-bb86-6724e97e27d7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:14.891281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.925811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:14.926493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:14.926771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:14.927605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:14.928317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:14.928621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:14.930345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:14.940909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 254/504] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:14.941682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:14.998044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b813868-f718-4c68-b862-a1c11fecd04c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.030593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.031342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.031632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.031968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.032671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.033177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.034385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece5ad-510b-469b-9c34-755bb2dd049b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.046034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 251/505] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.047361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.047816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.048049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489f9c9-e486-457a-8822-b2381d480577 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.048453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.064550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:15.065911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:15.081495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.082315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.082605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.082886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.093368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.094850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:15.112121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.113059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.113908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:15.113908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:15.113908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.113908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:15.113908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 255/506] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.114209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.126146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.132041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.140146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 252/507] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:15.141291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.240651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14c8ba-aace-4a17-b848-3149e3e6656f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.270028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cfb2a-7d67-499a-ae1f-d39c45e5d796 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.274503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.275243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.275600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.275966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.276772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.277141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.278929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.290120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 256/508] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.290972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.312398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.312398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.312398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.312398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.319716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.321183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:15.335823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.336758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:15.337644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 253/509] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.338897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.366462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.395220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.397504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba239-bb65-44c4-a19c-8c826ad0151a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.432425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.433255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.433545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.433828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.441711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.443795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.448481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601174f-37c1-4deb-9315-d68d836f2df0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:15.459120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.460177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:15.461104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 257/510] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.461649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.485573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.486299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.486592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.486871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.487597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.487912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.489291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.500954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 254/511] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.502035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.565154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de17aa-83a8-4ef6-9fd0-45c7d9550343 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.593350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.598994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.599784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.600071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.600350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.600506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697d12-f79a-4bb7-9c10-fc52c202866e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.610514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.611703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.613292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:15.631192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.632061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.632900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:15.632900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:15.632900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.632900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:15.632900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 258/512] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.633160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.639437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.640482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.640766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.641042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.651845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.653276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:15.670522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.671516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 255/513] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.672562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.735926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f1729-923f-408a-8cab-250e53031116 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.769432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.770123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.770412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.771287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.772131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.772444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.773319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784dc3-3e57-4525-a770-713365503091 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.773876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.790371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 259/514] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.791189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.808530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.809193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.809479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.809757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.810626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.810941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.812376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.832372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.848887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 256/515] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.849771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.870116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:15.893699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca18c3-3eeb-4b74-a5e6-8cdc50ae472e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.926980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.927697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.927986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.928258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.934856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.936298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.946606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59834bc-c4a2-4a42-93d1-52b49ca1d53f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:15.950864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:15.951767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:15.952644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 260/516] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:15.953503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:15.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:15.984733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:15.985046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:15.985328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:15.986080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:15.986391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:15.987802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:15.998881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 257/517] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:15.999680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.057429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53f16f-eb24-4cc5-b290-9b890f1625b4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.059049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.081570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.091683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.092341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.092854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.092962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.093599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:16.093901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.095336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.101167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436512f-90e5-4e75-beb2-47c021c46d1b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.106720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 261/518] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:16.107506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.135307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.135943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.136224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.136496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.203342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.204754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.204854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00b8952-254e-485c-a50f-b58c112f4342 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.238529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.239186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.239981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.240257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.270363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.283125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.303041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 258/519] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:16.303863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.308301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.309778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.402759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c9886-5023-410c-9cd9-604546a79369 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.413154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 262/520] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:16.414000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.441267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.442089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.442377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.442651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.443447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:16.443761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.445198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.462220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 259/521] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:16.462945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.476020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.503449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.515227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e57ec-8d37-4259-bf12-f5244307899e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.548070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.548735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.549019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.549294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.550056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.551525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.562999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447758f3-ec06-4b9c-9555-ce6c3eb01735 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.570737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 263/522] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:16 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:16.571539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.611824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.612647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.612934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.613215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.622379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.623903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:16.639460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:16.640727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:16.641615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 260/523] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:16.642901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.692406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476859ab-e9ba-4dd8-825f-57c395d8f6d8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.714083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.735697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.736616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.736972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.737326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.749810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.751316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.757160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:16.760726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bef3c7-2b02-4f37-9911-87750f4fe3a0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:16.769607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:16.770798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:16.771691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:16.771691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:16.771691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:16.771691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:16.771691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 264/524] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:16.771948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.811553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.812442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.812795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.813074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.826199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.827666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:16.845331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:16.846413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:16.847595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:16.847595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:16.847595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:16.847595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:16.847595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 261/525] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:16.847944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.893503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b6547-f6fd-4e27-b6cc-1decf23c038c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.927830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:16.928567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:16.928764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:16.929038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:16.929863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:16.930203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:16.931657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:16.945996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 265/526] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:16.945996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:16.978138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9e625-4e70-45aa-a88a-9c7b00025996 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:16.994716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.026185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.027404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.027961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.028244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.029368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:17.029688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.031987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.055881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 262/527] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:17.061469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.077881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.085609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa55359-b1f1-4c25-91b8-b796648095be None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.144542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.145209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.145496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.146332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.147051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:17.147353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.148737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.170230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 266/528] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:17.171145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.206937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefab645-dfe1-4cd6-b8d0-a3ff238f5c9d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.247262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.247863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.248227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.248508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.249252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:17.249757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.251065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.264152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 263/529] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:17.265639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.296461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.306032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c38ba-2577-4ddc-8821-29595095d21a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.343116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.357462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.358909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.359667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.360170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.377620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c52d1-9aaf-41f1-869f-2a6d1702f896 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.419201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.420957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.421528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.422031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.431988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.433897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.445089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.446695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:17.453781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:17.454644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:17.455561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 264/530] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:17.456341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.560767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.571460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 267/531] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:17.572459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.591789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52d360-d400-4bd9-ad49-ab329792490a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.626049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.634109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.635025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.635401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.635768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.643521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.645115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:17.659536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:17.660606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:17.661685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:17.663686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 265/532] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:17.664643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.696802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31570d0-a6c5-4768-b042-24167d9a71d5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.732859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.733756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.734185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.734558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.735630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:17.736045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.737648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.759209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 268/533] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:17.765169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.791321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09424c7b-6ea5-4443-8db6-8fb5ee0f09a7 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.832493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.834669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.835367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.835809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.836460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.852526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.855126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:17.878994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:17.880612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:17.881649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:17.881649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:17.881649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:17.881649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:17.887885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 266/534] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:17.887885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:17.893385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64566-5808-49c1-af46-2c039e7d5202 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:17.934253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:17.934981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:17.935243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:17.935519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:17.936351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:17.936664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:17.938124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:17.943538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:17.951232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 269/535] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:17.952097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:18.016805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782d687-45d6-4d1c-9e75-ff5c2ad52e42 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.052599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.053329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.053616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.053890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.054795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=0a4b93133aa146bca7ee0f205cf623a8) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:18.055166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.056672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.070663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 267/536] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/role_assignments?user.id=0a4b93133aa146bca7ee0f205cf623a8&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:18.071462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:18.081247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb8b76-4d6e-4945-9b9f-27b8a1696a18 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.099223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:18.136185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.136876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.137164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.137445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.138212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:18.138526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.142953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.175894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 270/537] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:18.183021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:18.312351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d327b-3d2f-4456-8c36-2c81001d6d2b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.323285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:18.346647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b906e-deb5-48f8-96fe-e597f950a075 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.356817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.357834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.358342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.358938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.359970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=069791ace6bc45a494dd77b732302466) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:18.360559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.362222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.376865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 268/538] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:18 2022] GET /identity/v3/role_assignments?user.id=069791ace6bc45a494dd77b732302466&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:18.386093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.386960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.387249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.387528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.395414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.397412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:18.412776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:18.413697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:18.414584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 271/539] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:18.420171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:18.532978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694b8e-b1e5-485b-810b-2aeb786efd81 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.570552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.571235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.571528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.571808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.572632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:18.572949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.574471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa88a02-d5d7-4920-8ea8-b143e2c8941d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.588902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 269/540] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:18.594760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:18.632711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:18.707367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2874e30-b4d8-4793-b803-daf53ec9d15a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:18.754875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:18.756124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:18.756418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:18.756701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:18.757558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=fd6ce0d979904b6e9db4968604e0192b) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:18.757876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:18.759350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7dd8f-fdef-48d7-aadb-a2077a79bad1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:18.766082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 272/541] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:18 2022] GET /identity/v3/role_assignments?user.id=fd6ce0d979904b6e9db4968604e0192b&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:19.068334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510cc7-4d41-4c58-9cb1-fbea5650171e None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.068852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510cc7-4d41-4c58-9cb1-fbea5650171e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.068962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510cc7-4d41-4c58-9cb1-fbea5650171e None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.070514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 270/542] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:19.076867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0561-73e7-424d-83ef-510e72889989 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.077300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0561-73e7-424d-83ef-510e72889989 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.077474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c0561-73e7-424d-83ef-510e72889989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.094627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-944c0561-73e7-424d-83ef-510e72889989 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:19.095821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-944c0561-73e7-424d-83ef-510e72889989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:19.180462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 273/543] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:19.209597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e601f98-1a8b-45db-9024-8a28ca316f88 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.209884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e601f98-1a8b-45db-9024-8a28ca316f88 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.210188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e601f98-1a8b-45db-9024-8a28ca316f88 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.211687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 271/544] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:19.217155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.217451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.217729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.224610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e601f98-1a8b-45db-9024-8a28ca316f88 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.250010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:19.251212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:19.277743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:19.277984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:19.371244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e601f98-1a8b-45db-9024-8a28ca316f88 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.410491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:19.412198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.412699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.413184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.426068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 274/545] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:19.433215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:19.445999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:19.447103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:19.448089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:19.450341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 272/546] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:19.455793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.464186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.621784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fca61-5d79-4e1d-9b38-11f36699d772 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.631481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:19.645452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e33bd6-6fa9-4d9a-81f4-687e6d3d260d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.658323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:19.659388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.659809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.660202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.672263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:19.680901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:19.681929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:19.682925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:19.689606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 273/547] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:19.691003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.718669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:19.719359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.719646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.719925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.720681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:19.720990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:19.722414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:19.731893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 275/548] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:19.738913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.782147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:19.806969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7f63-e657-446c-9185-7b6e683f8038 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.843086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:19.843779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.844067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.844348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:19.845098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:19.845408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:19.846953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:19.860818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 274/549] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:19.862030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:19.878382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:19.928344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4cbcd-9226-488f-b6f0-38dc5d0009dd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.968045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c0c2-5f2e-41e4-ab93-8f66567a6455 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:19.995570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:19.996383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:19.996668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:19.996948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.004810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.005644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.005935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.006246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.017039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.018573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.020400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.022440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:20.045534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:20.046473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:20.047302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:20.047302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:20.047302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.047302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:20.055763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 275/550] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:20.056686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:20.058870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:20.059880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:20.060831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:20.060831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:20.060831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.060831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:20.060831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 276/551] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:20.066853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.146504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:20.185251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c216e-d39c-49b1-afe9-7195d361b75a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.223269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.224109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.224402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.224684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.225520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:20.225840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.227269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.240301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 276/552] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:20.245905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.259011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:20.315943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f90adc-9d3b-4fcf-8233-a1e09e7fff55 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.354462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d39610-8974-4793-beb3-a71214997b1d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.380889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:20.387701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.388224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.388509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.388786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.389519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:20.389832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.391505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.394638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.396209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.396737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.397199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.405447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.407274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.418393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 277/553] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:20.422963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:20.423933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:20.424851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:20.425360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.431435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 277/554] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:20.433469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.543024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2571a-0a3d-41a6-82e0-88d1ccfcdc07 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.582290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac7a33-493c-422c-93aa-52e4fbe5e139 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.582928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.583998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.584586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.585068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.586141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:20.586673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.588491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.612016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 278/555] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:20.613643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.635239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:20.652375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.653109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.653395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.653672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.665585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.667005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.669729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:20.701702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:20.702824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:20.704621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 278/556] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:20.708628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.737218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc5446-3e33-4228-b6a0-1fee8c774b13 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.775150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.776314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.776823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.777307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.795496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.798285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ea1a0e2f-657b-4e91-8807-6009bba66629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:20.832581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:20.833593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:20.834644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:20.834644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:20.834644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:20.834644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:20.841980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 279/557] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:20.841980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba6bd-6f49-4401-8853-75fb08d6a2a0 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.841980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba6bd-6f49-4401-8853-75fb08d6a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.841980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba6bd-6f49-4401-8853-75fb08d6a2a0 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.841980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 280/558] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:20.847493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.847785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.848069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.867162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:20.868471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:20.905044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56f04c-0bf2-4970-a8aa-22bc74f20155 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:20.949738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:20.954034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:20.954582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 281/559] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:20.955306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:20.978124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 279/560] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:20.982707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.002854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.058217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.088697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d676-a3f6-4074-b77a-c0698158d318 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.125816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.126726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.127011 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.127288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.128033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:21.128278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5050f1-1374-41d2-a1a7-e077ae00d816 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.128385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.129796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.150599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 282/561] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:21.152722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.185909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.187083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.187708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.188184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.249726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.267174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d806cfc-faa3-4c0a-b158-61e3217985d0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.278518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.280672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6e88c-4c72-4d9d-808d-d4b8b0caede8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.303376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.304519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.304519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.304823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.315968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:21.324718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:21.325690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:21.326566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:21.330038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 283/562] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:21.331112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.339565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.391584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 280/563] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:38:21.392948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ab6-62e4-46de-b0e9-11778a3cc784 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.393240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ab6-62e4-46de-b0e9-11778a3cc784 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.393541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1799ab6-62e4-46de-b0e9-11778a3cc784 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.398190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 281/564] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:21.399108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799ab6-62e4-46de-b0e9-11778a3cc784 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.431783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef808f9-6293-42c2-babf-184f2e9794af None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.464861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.465597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.465885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.466192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.472914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.474439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-c4fdeb44-0950-4faa-aa0a-f2949fea1692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:21.488479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:21.489938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:21.490889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 284/565] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:21.491575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fb8f1-4832-4c81-9ad1-3daaad56e00f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.491676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fb8f1-4832-4c81-9ad1-3daaad56e00f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.491950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fb8f1-4832-4c81-9ad1-3daaad56e00f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.493208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 285/566] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:21.497883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fb8f1-4832-4c81-9ad1-3daaad56e00f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.499526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1799ab6-62e4-46de-b0e9-11778a3cc784 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.501333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.533664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.534362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.534650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.534934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.535673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:21.535984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.537370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084100c9-c5fb-45d0-9c27-9f8af43bf0ff None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.545194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 282/567] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:21.546401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0550b-1219-4372-8464-52703835298e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.546684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0550b-1219-4372-8464-52703835298e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.546964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0550b-1219-4372-8464-52703835298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.563321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-dad0550b-1219-4372-8464-52703835298e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:21.564531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dad0550b-1219-4372-8464-52703835298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:21.572325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.599086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fb8f1-4832-4c81-9ad1-3daaad56e00f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.630721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 283/568] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:21.631548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0550b-1219-4372-8464-52703835298e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.649019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.649814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.650137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.650418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.661871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.663374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-be46441c-ba86-4dad-937d-df16f215bcb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:21.681511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:21.682604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:21.683502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:21.683502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:21.683502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:21.683502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:21.683502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 286/569] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:21.684050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.684167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.684449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.702439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:21.703632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:21.732399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0550b-1219-4372-8464-52703835298e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.754822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.765925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.766616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.766905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.767182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.767970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:21.768279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.769719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.771650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 287/570] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:21.772457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.801405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 284/571] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:21.801405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.801405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:21.888808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81f60c-be58-4288-bd71-a81739cb2c7f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.901007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4812a0-d6a5-4076-a863-ca341920a748 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:21.923864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.924658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.924934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.925204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.934707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:21.935178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:21.935462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:21.935734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:21.936509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:21.938488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:21.945901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:21.953509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:21.954650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:21.955589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 285/572] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:21.956774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:21.956857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:21.958240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 288/573] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:21.960142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:21.999035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.024088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.060084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970ed6-38bc-45fe-aeed-cec3a2540506 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.065639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b361d-2bbb-424c-8c7a-a74e3a5eb799 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.093977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.094800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.095084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.095362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.099499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.100156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.100448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.100716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.101433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:22.101738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.103154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.106145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:22.114039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:22.115146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:22.116013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:22.116644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 286/574] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:22.116724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 289/575] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:22.116793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.116869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.212463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.216888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c7f3-7612-4a8f-bd47-2c2f05564cd0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.226230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907496f1-c397-4760-b45f-f71ae90e41c7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.249518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.253856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.254560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.254851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.255131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.255879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:22.256193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.257698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.261141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.262389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.262853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.263337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.266093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 290/576] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:22.266904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.279246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=8344a4e41ea548568e99784e3a6ef493, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.280691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.293919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 287/577] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:22 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:22.294765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.389770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49311d7-9922-48f1-90b7-0ebab03b1ab5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.424011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a372d4-4086-4e95-8068-c69a4fc4cba3 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.432517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.433202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.433487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.433764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.434721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:22.435097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.437597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.450332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 291/578] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:22.451191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.469205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.470045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.470328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.470600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.477455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.483429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.484386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:22.500261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:22.501149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:22.502071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 288/579] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:22.502558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.535109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.575208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100be79-e0ce-4707-b8ce-cae9db87e36d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.609561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbd6de-007c-4f9b-90b2-78649d8e7709 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.618001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.618669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.618949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.619667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.620411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:22.621121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.622840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.638756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 292/580] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:22.638756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.644994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.645838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.646154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.646434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.657585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.659078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:22.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:22.679427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 289/581] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:22.680410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.732248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.752874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b62c06-0b8c-4bcf-b52d-b880feb5606b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.772567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.789063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8f9b7-783d-459b-bfe8-80f7dd9690fa None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.792764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.793567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.793854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.794162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.803858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.805311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:22.822326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:22.823319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:22.824261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 293/582] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:22.825357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.831123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.831792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.832079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.832351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.833723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:22.834110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.836030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.850329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 290/583] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:22.855699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:22.926796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e54-df51-4cd2-812d-51a869bdec39 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.960277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:22.961075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:22.961361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:22.961641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:22.964274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d48bc7-b496-45e7-bd0d-11c6185b1103 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:22.972449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:22.974561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:22.976416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:22.992539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:22.993645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 294/584] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:22.994604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.009295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.009905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.010218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.010500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.011355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.011787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.013167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.030805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 291/585] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.030805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.032310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.108461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d86224-7280-4054-8a15-d031f58bcb54 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.133164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365cacd-a6d1-492c-9415-e1bb6d4b56c1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.143136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.144159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.144452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.144733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.146474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.146792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.148279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.170267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 295/586] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.170267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.176147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.176885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.177595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.177878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.179140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=ef2bef098cce41279cc9a5d81f0dd782) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.179602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.181397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.198341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 292/587] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/role_assignments?user.id=ef2bef098cce41279cc9a5d81f0dd782&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.198341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.232293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.293262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.299040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34094d-b1fb-481b-abc5-c7bce82a66d1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.334405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57aacea-7c76-43d6-82e9-e1f5afcba24a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.338300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.339171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.339598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.340013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.340907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.341371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.343145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.361226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 296/588] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.362469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.374813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.375891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.376280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.376664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.386251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.387851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:23.405276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:23.406376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:23.407889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:23.410065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 293/589] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:23.410942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.476775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba6bd-0617-4385-b278-a1058f2f144e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.503444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.533571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.534773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.535271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.536391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.550454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e894dd-ec9a-4285-9531-bf611b2ebda3 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.551463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.554878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:23.578308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:23.579300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:23.580225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:23.580225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:23.580225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:23.580225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:23.583050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 297/590] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:23.583802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.585172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.597182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.598029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.598869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.599173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.610036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.611521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:23.631688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:23.632638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 294/591] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:23.634217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.702614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017707eb-02a5-4bef-a247-cd4ae16817a1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.740079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.741091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.741685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.742214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.743411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.743974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.745773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.765699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 298/592] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.765883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c0333-c622-4138-bbad-306abec1177c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.767012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.793714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.813811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.814916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.815206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.815489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.816221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.816565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.819279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.839595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 295/593] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.840815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.885513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d71bb-ec62-4497-b791-93a7746e15f2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:23.906915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:23.928216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:23.928888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:23.929168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:23.929489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:23.930370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:23.930725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:23.932675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:23.951566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 299/594] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:23.952529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:23.975498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb2d7e-221c-40af-8877-80d8d91178bd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.022840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.023640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.024652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.024929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.042547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.045089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.046250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.068293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd0543-bf39-4fc1-8185-5ef7d49c9349 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:24.072059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:24.072927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:24.073805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:24.073805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:24.073805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:24.073805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:24.076269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 296/595] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:24.077312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.105895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.106768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.107052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.107323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.115191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.117051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:24.131611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:24.132541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.133426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:24.137591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 300/596] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:24.138569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.207273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.210893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383088cd-0521-4100-ad83-408aec4cd10d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.257791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.258748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.259099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.259444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.260358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5c7e-d056-4492-a817-38f9083a5ec1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.270372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.271922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:24.291437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:24.292418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:24.293297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:24.293297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:24.293297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:24.293297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:24.293297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 297/597] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:24.294029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.309065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.310226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.311101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.311617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.312926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:24.313508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.315915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.330869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.337659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 301/598] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:24.339552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.426186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc018af5-0b7f-4894-ba63-bf7758b766cb None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.456143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26cb6a-2cda-4a60-a495-5c718158e5ff None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.469812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.470907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.471351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.472166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.473395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:24.474440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.476593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.492609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 298/599] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:24.493648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.499147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.499886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.500235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.500574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.501381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:24.501749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.503281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.511317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.522063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 302/600] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:24.522063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.585780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.610231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee82203-2dd1-4bf7-8392-4a49b5e5a293 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.629327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2494f-8e7d-40d6-aa8b-aab23cabc2f4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.643941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.644726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.645005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.645297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.653245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.654726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.666887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.667977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.668459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.668926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/0a4b93133aa146bca7ee0f205cf623a8/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0eeee19e-44a0-46bf-8e9a-d8966f35ce8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:24.669098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:24.669926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.670870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:24.675023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 299/601] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:24.676303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b1aa3-bc97-4d3c-9c6c-2362ed3d4c5b None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.676615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b1aa3-bc97-4d3c-9c6c-2362ed3d4c5b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.676894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b1aa3-bc97-4d3c-9c6c-2362ed3d4c5b None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.678307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 300/602] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:24.678921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b1aa3-bc97-4d3c-9c6c-2362ed3d4c5b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.689235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=0a4b93133aa146bca7ee0f205cf623a8, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.690730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.704596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 303/603] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:24 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/0a4b93133aa146bca7ee0f205cf623a8/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:24.705461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:24.774037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.812419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b1aa3-bc97-4d3c-9c6c-2362ed3d4c5b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.859463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.860262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.860550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.860828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.868715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8bccc-2cd5-4dad-bf55-f5ef3cbefa9a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:24.870405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.872149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.884371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-12ec4b06-7b44-497e-b40e-aeb22261385b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:24.892665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:24.893621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:24.894560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:24.898098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 301/604] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:24.900675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.902823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.903318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.923716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:24.925078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:24.928286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:24.928441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:24.929395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:24.929395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:24.929664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:24.930020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:24.931467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:24.946412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 304/605] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:24.947893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.017989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 302/606] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:25.017989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.090913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:25.092394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e6024-9115-4277-a314-f591ee8702c1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.129131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.130733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.131245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.131739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.132725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:25.133248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:25.134911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:25.140444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6d999-509b-4930-b5cb-844496d5c7a3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.153763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 305/607] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:25.155105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.183435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.184566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.184852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.185128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/069791ace6bc45a494dd77b732302466/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.188820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:25.205615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=069791ace6bc45a494dd77b732302466, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:25.207757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:25.230492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 303/608] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:25 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/069791ace6bc45a494dd77b732302466/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:25.232045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.328446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6fb4e-55d7-40cb-be4a-bf59acf780ab None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.399743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.400496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.400803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.401051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.414490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55a832-2936-4ed7-b5c0-8bb4934d057e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.427700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:25.437886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:25.438917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:25.439797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:25.447244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 306/609] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:25.450326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:25.456541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.481668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.483171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.483515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.483862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/fd6ce0d979904b6e9db4968604e0192b/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.503577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=fd6ce0d979904b6e9db4968604e0192b, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:25.505084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bcece-a805-47c7-a0bf-80b685b19cb6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:25.522972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 304/610] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:25 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/fd6ce0d979904b6e9db4968604e0192b/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:25.524851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bce4f-ae2f-4ec9-9ef4-80172aeb75c0 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.525202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bce4f-ae2f-4ec9-9ef4-80172aeb75c0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.525499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bce4f-ae2f-4ec9-9ef4-80172aeb75c0 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.527224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 305/611] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:25.533400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.533700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.534017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.556347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:25.557612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:25.644398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 306/612] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:25.662409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:25.670465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.682121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7f8a-8f07-4e05-b4bf-b16083987b3c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.782724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.783821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.784302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.784783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.785543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:25.785853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:25.787953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:25.792495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:25.808081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a08057-65ac-482a-85bf-d505c7a6eaa2 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.808903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 307/613] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:25.810029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.858617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:25.860429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:25.860716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:25.860996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:25.875735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:25.886432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:25.887398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:25.888224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:25.888884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 307/614] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:25.897104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:25.945907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ed194-72e4-4c49-85ff-330aea999ad2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:25.998936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.000120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.000609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.001088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.014769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.016874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.029746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af4406-a8e8-4499-9445-1e74d8edf83e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:26.035443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:26.036335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:26.037236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:26.037236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:26.037236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:26.037236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:26.039464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 308/615] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:26.045371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.051450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:26.092974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.094832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.096284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.096788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.098238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:26.099744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.101872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.120688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 308/616] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:26.126701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.160371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:26.253033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d39a7-1d36-4b02-b730-21c455ee1ead None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.306090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.306090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.306090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.306090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.306498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:26.306498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.306498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.319368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 309/617] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:26.325148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.340160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f035860e-27d4-4b6a-9ee3-b68a98cadb98 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.402425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:26.433889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2a504-d55e-468d-83c6-ea0ffb078d5c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.439228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.440046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.440328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.440607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.470490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.472199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.472894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.474311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.474704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.475027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] PATH_INFO: `/v3/users/nova` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.482564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.484093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:26.500439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:26.501396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:26.503019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 310/618] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:26.507227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:26.509025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:26.509856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:26.511501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:26.511501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:26.511501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:26.511501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:26.511501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 309/619] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:26.518887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.614437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979bd7-3a47-402b-b498-58da9e2ff89e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.641979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:26.660867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.662468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.662607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.662770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.663610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:26.663918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.665373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.681926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:26.685072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 311/620] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:26.690790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.726544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef255c-a704-424e-9332-ef39417a4acf None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.762487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.762673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.762921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.763151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.763868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:26.764174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:26.765821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:26.779233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 310/621] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:26.784507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:26.856268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a646ca0-349a-4c04-a940-8bb8ab39cdee None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:26.900136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:26.900787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:26.901067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:26.901339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:26.982109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8905d-d9d3-4f6c-8cc5-00dae448eb3b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.011500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.022617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.023401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.023684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.023979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.024688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.045758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.047378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:27.072947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:27.073920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:27.075098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:27.076144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.077649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.082323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 311/622] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:27.094985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.191741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 312/623] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 10:38:27.198934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.294039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c7984-242f-45a8-aed0-602427db9ff7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.308958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.330279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54573a8-bcdc-4a58-80c5-2331f1af0e55 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.336380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.337058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.337344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.337624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.338927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:27.339240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.341063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc155-868a-4e48-8e00-488b4cca0d2c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.366046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 312/624] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:27.367427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621a85-9fa4-429d-813d-455e662039f0 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.367714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621a85-9fa4-429d-813d-455e662039f0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.367988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621a85-9fa4-429d-813d-455e662039f0 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.370460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.371274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.371557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.371831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.373344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 313/625] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:27.374281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16621a85-9fa4-429d-813d-455e662039f0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.384396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.386380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-2dde9af2-7091-4195-9b55-cc72913808db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:27.410885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:27.413804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 313/626] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.415616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.416417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.429189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.444530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:27.446294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:27.489619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16621a85-9fa4-429d-813d-455e662039f0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.526233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 314/627] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:27.527678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.531920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.532940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.533419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.533864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.549755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.551166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:27.570554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:27.571483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 314/628] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:27.572846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.619661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.678914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d6ec3-ba95-4b3b-8b77-98ed9405c97e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.684732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.691368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e895f-7dc2-459a-8e8a-9b245631a03c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.726690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.727480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.727765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.728045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.728770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:27.729080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.730599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.736636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.738274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.738713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.739187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.747458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 315/629] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:27.747458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.758306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:27.769215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:27.770898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:27.771733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:27.772374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 315/630] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:27.772933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.862088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb3a2-4a0b-44c3-a6b2-df91b63b5a77 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.891078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:27.898031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab9620-042c-42b8-9f56-28f73677cb42 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:27.904757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.905803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.906122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.906408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.907520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:27.907834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.909837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.922087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 316/631] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:27.922801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.936769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:27.937779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.938084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.938380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.945477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:27.946954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-12134b92-dab4-42fd-b1f1-b9854455afc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:27.961292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:27.962255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:27.963154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:27.967151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 316/632] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:27.968298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45733da-4616-4140-aed5-7d03763c93dd None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:27.968590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45733da-4616-4140-aed5-7d03763c93dd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:27.968870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45733da-4616-4140-aed5-7d03763c93dd None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:27.971416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 317/633] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:27.971416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45733da-4616-4140-aed5-7d03763c93dd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:27.994783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.036274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16ad13-2fb0-4be4-9b34-6c99139c204d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.071501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.072164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.072447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.072728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.073466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.073774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.075188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.088040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 317/634] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.088884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.090257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45733da-4616-4140-aed5-7d03763c93dd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.126745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.128254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.128722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.129177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.135040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.137576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.139066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-063abc49-0387-41b0-a78b-1b9ba00fa246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:28.152840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:28.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:28.154697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 318/635] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:28.155644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.155903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.156178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.173532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:28.174761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:28.195968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c843-ad49-41d9-adee-cdf5afba53df None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.227854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.228509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.228788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.229060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.229835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.230188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.231650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.243653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 319/636] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:28.244510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.246551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 318/637] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.247370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.259613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.349635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb3859-2358-4273-95f5-8fe21933378d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.350629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc9e9b-3526-49f5-87ac-95065903bb79 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.356663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.383807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.383964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.384395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.384673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.384864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.384955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.385039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.385218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.385716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.386050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.387498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f22b2-8f17-4a28-8083-bf08f573b334 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.395228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.396639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.399103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 319/638] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.400346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4591f4-313b-4bcc-aea5-fd9fc7aa12ed None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.400642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4591f4-313b-4bcc-aea5-fd9fc7aa12ed None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.400924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4591f4-313b-4bcc-aea5-fd9fc7aa12ed None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.402188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 320/639] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:28.402886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4591f4-313b-4bcc-aea5-fd9fc7aa12ed None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:28.416057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:28.416999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:28.417873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:28.417873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:28.417873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:28.417873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:28.417873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 320/640] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:28.418171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.476681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.510635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4591f4-313b-4bcc-aea5-fd9fc7aa12ed None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.517505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6bb529-aab2-4576-ab36-3217bd548676 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.543599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232180b-482e-47c5-8387-d07648784bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.544248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232180b-482e-47c5-8387-d07648784bab None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.544537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232180b-482e-47c5-8387-d07648784bab None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.544819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232180b-482e-47c5-8387-d07648784bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.545646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232180b-482e-47c5-8387-d07648784bab None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.545980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232180b-482e-47c5-8387-d07648784bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.547422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232180b-482e-47c5-8387-d07648784bab None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.550889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.552231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.552509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.552781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.553801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 321/641] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.554929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232180b-482e-47c5-8387-d07648784bab None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.563812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-3b2956d7-7e87-4c3a-bdb5-55792f5dfa07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:28.571759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:28.572880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:28.573739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:28.575259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 321/642] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:28.576716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.577004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.577278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.594811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:28.595558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.596035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:28.675145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 322/643] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:28.675941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.690872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232180b-482e-47c5-8387-d07648784bab None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.730185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.730416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.731879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.731879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.731879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/ef2bef098cce41279cc9a5d81f0dd782/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.750634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=ef2bef098cce41279cc9a5d81f0dd782, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.752208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.767500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 322/644] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:28 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/ef2bef098cce41279cc9a5d81f0dd782/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:28.768360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.795947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21136788-33df-43e2-a0cb-0b87c9a74276 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.851544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.852284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.852572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.852851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.853620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.854121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.855534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.868194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 323/645] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.869089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.879917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:28.903161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e0a2c-5dcf-4956-a11e-832f9c5502cf None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:28.941577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:28.942239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:28.942543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:28.942824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:28.943598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:28.943908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:28.945336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:28.954751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 323/646] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:28.955664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:28.998896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f517cc-dcb6-4ed2-8073-b758c1c0e6fd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.010107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.047108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.047969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.048256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.048533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.063423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:29.074651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:29.075781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:29.076960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:29.083196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69cb61-9162-4be6-b5ce-a84962aef439 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.083354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 324/647] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:29.083354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.118211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.119156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.119543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.119923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.128778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.130608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:29.146289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:29.147431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:38:29.148567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:29.151252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 324/648] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:29.151530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.154625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.210624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfd4c1-3a69-439c-9cdd-6e246808a305 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.264373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.264634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0b986-4b1e-4258-9de6-ab27c4b9c8ce None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.265436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.265717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.266018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.282400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:29.305545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:29.306705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:29.307823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:29.307823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:29.307823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:29.307823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:29.309008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.309976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.310132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 325/649] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:29.310266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.310702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.311410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.311632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:29.312049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.313528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.328317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 325/650] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:29.329347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.339345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.407961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.431862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13271-495a-4794-a753-6f26384ef48d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.458786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f3e1-dee1-4bce-b47f-1ad0d4a7193c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.472888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.473611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.473899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.474209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.475058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:29.475375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.476940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.496358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 326/651] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:29.497306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.506192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.507738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.508259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.508750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.510174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:29.511112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.513639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.533214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 326/652] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:29.534506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.602393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.633990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6924e94-cb5b-44cc-94e4-35dfc2ee8040 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.661686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d381-c857-4101-acb3-bc8643aa259e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.681211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.682620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.683462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.683946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.692650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.696655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.698332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.706161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.707027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.707636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.708119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:29.716886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:29.717789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:29.718823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:29.718823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:29.718823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:29.718823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:29.722807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 327/653] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:29.723882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.786900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.788723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.827195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd64957-3cea-4fa6-882f-a2c5bc95e943 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:29.856308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:29.862539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:29.863458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:29.865117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:29.865359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] PATH_INFO: `/v3/users/glance` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:29.873012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:29.874513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:29.890373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:29.891371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 10:38:29.892324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:29.896410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 328/654] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:29.897334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.906430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 327/655] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:29.907535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:29.934332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.005785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e7790-b278-4829-8b0a-0f54505af57b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.017867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11624a3-c3fb-44d3-afb2-6e5d77a5756f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.041945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.042631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.043050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.043334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.044051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:30.044357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.045721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.056436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.057387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.057897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.058424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.059483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:30.060015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.061677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.062098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 329/656] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:30.062958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.086115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 328/657] 10.222.0.36 () {56 vars in 1137 bytes} [Mon May 23 10:38:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:30.087324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.115186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.172595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdfd8-4a5b-4cc0-8891-a66eca132089 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.211224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.212032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.212316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.212589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.225103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.226608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:30.245516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:30.246512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:30.247405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:30.247405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:30.247405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:30.247405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:30.247405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 330/658] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:30.248111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.310311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.332665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b11f4-d286-4034-b107-f6e96244ba08 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.371668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4941c-17c2-4c80-ba85-fb1dcf1ef181 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.373354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.375438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.376337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.376739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.377872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.386891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.388563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:30.404681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:30.405884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 10:38:30.407101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:30.408915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 329/659] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:30.412483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.418030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.418695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.418979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.419253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.498166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.499639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.518377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8acf96-2b37-4b50-9638-822f0f02b121 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.555840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.556511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.556804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.557085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.558141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:30.558494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.559885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.573750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 330/660] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:30.574555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.577213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.607630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 331/661] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:30.609230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.612797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.689584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73da27-de7e-4d82-afdf-da9e99b77d96 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.725854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.726820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.727278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.727703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.728709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:30.729172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.730783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.734381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64050bd-a2b6-4065-abea-3f116870161b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.756643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 331/662] 10.222.0.36 () {56 vars in 1139 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:30.756643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.793113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.794610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.795994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.796479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.806266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.808713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.821475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:30.830835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:30.831697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:30.832630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:30.835571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 332/663] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:30.837410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.868988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7270d-e23b-49fc-ab3a-3386ff7de700 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:30.909616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:30.910502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:30.911336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:30.911620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:30.923918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:30.924654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:30.927350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:30.948222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:30.949183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:30.950081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:30.950081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:30.950081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:30.950081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:30.953921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 332/664] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:30.954919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:30.972407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b97ea-bac2-4375-9f77-790f29bf4227 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.009055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.009708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.010023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.010299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.060068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306d6c-9f70-489b-97cc-418241889e3f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.074047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.081296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.083253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f7ac1-fd97-47c1-bdb4-50ae8017b9f4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.095578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.096230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.096511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.096785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.097555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:31.097859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.099938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5633c4-df04-4184-b9f8-8467ec0e9eac None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.110995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 333/665] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:31.112194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18f024-faec-4f9e-a586-0cc6f0d5a544 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.112482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18f024-faec-4f9e-a586-0cc6f0d5a544 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.112757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18f024-faec-4f9e-a586-0cc6f0d5a544 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.114027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 334/666] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:31.114718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18f024-faec-4f9e-a586-0cc6f0d5a544 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.147005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.186299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 333/667] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:31.187810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a80bfc-7b31-4a84-8b39-5add9475091f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.188106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a80bfc-7b31-4a84-8b39-5add9475091f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.188381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a80bfc-7b31-4a84-8b39-5add9475091f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.189800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 334/668] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:31.193919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a80bfc-7b31-4a84-8b39-5add9475091f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.216203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18f024-faec-4f9e-a586-0cc6f0d5a544 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.251985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.252654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.252941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.253217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.253931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:31.254274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.255674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936f4c-f869-4edd-ab10-998686d57be7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.267405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 335/669] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:31.268573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae7c6c-01a7-4072-923b-d2449044e1a9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.268869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae7c6c-01a7-4072-923b-d2449044e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.269147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae7c6c-01a7-4072-923b-d2449044e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.285063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-54ae7c6c-01a7-4072-923b-d2449044e1a9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:31.286914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-54ae7c6c-01a7-4072-923b-d2449044e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:31.297638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a80bfc-7b31-4a84-8b39-5add9475091f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.311486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.330884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.331671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.331892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.332161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.333511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=0a4b93133aa146bca7ee0f205cf623a8) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:31.333819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.335271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.340894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 335/670] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/role_assignments?user.id=0a4b93133aa146bca7ee0f205cf623a8&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:31.341659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.363573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 336/671] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:31.363573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.363573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.363573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.372587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:31.373884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:31.387730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.461986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 337/672] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:31.461986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.486524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbc158-209a-4ca6-a282-edb16b472488 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.527096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.527482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.527727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.528000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.539730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.541178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:31.559502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:31.560426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 336/673] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:31.561494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.568225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.579361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937d2a2-85fd-4f1b-942c-045160e847db None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.614271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.614983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.615270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.615545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.622190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.623637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:31.637510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:31.638722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:31.639648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 338/674] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:31.642760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.652268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.675579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b25cd-7f85-48dd-9552-21e059125670 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.716134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.717264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.717746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.718273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.729325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:31.737162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:31.738173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:31.739257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 337/675] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:31.743088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.752337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6acf5-3809-419a-99bd-e4eb3f13931a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.790854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.791752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.792053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.792448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.803901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:31.812114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:31.813094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:31.813942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 339/676] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:31.814716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.833589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.855013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c019cf-4cdd-41cf-b88b-afd64e0a15d8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.890510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.891103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.891386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.891662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.892384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:31.892695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.894084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.901371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:31.911592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 338/677] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:31.911592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:31.933248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17f6e-64d1-4752-a9c8-047166d8937c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:31.968183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:31.968844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:31.969127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:31.969403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:31.970213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:31.970521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:31.972173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:31.990049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 340/678] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:31.990049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.021276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e816c0-13cb-440d-83ab-191ed1bdde01 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.056206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.057114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.057443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.057716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.058525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.058830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.060225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.063202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.073875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 339/679] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.073875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.094843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8577a-7d8c-45b2-9711-5d0faf5e2ad1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.132243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.132908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.133195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.133470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.134642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.135368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.135764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.137350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.151863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 341/680] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.152535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.175672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ce853-20ee-462a-a75e-4bc00326e72d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.209296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.210113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.210399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.210692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.217132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.218621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:32.233628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:32.234567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:32.235460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:32.240316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 340/681] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:32.241160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.266569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21ec38-67bd-4ef7-8431-9ba4d34cf093 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.307689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.308611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.308611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.308892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.309719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=069791ace6bc45a494dd77b732302466) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.310113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.311560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.313519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.325992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 342/682] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/role_assignments?user.id=069791ace6bc45a494dd77b732302466&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.325992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.359195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce9018-6031-430e-a7bb-d0b27602b35c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.392151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.413545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.414310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.415158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.415483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.432941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.435046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.443280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c68a47-6077-4e2a-b022-6b1e95dd0da3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:32.459317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:32.460431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:32.461315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:32.461315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:32.461315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:32.461315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:32.461315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 341/683] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:32.462571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.492580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.493483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.493773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.494082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.506448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.508199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:32.527370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:32.528319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:32.529199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:32.529199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:32.529199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:32.529199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:32.529199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 343/684] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:32.529924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.595400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.603547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc284f-ea2a-4901-8059-d41dcc29127b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.653669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.655156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.655662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.656134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.657116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.657620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.659310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.661849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bce08a-4785-4782-875d-b58fed17a7c0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.672622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 342/685] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.673707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.696109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.696778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.697067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.697344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.698120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.698490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.699809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.716614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 344/686] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.717509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.717617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.784670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7a4f8-1f20-4354-acac-d070c53632d7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.824127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.825225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.825757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.826375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.827289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.827807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.829523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.839333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88f400-de29-413c-bfc2-3b79fb2eee7b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:32.843905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 343/687] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.844960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:32.855802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:32.888264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:32.888920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:32.889203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:32.889479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:32.891139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=fd6ce0d979904b6e9db4968604e0192b) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:32.891460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:32.892912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:32.909983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 345/688] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/role_assignments?user.id=fd6ce0d979904b6e9db4968604e0192b&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:32.909983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.019828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.045035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862403d-3ec6-41a8-b8c0-5c3274fb4736 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.063033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466808de-52f2-4be6-99ae-4244cb834544 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.104487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.105924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.106234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.106665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.115697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.117312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:33.134188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:33.135174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:33.136085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 346/689] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:33.141011 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.143134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.144144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.144585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.145015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.161787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.163320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:33.179768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:33.180758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:33.181691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 344/690] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:33.186904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.192919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.291231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837fc5b4-d9d2-4266-98c7-f47b558db762 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.304623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d81f9-f1ba-4f6e-a72c-609564611b23 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.330323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.331061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.331352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.331633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.332740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:33.333062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.334547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.361941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 347/691] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:33.371268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.374735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.375419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.375706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.375983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.376837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:33.377148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.385465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.422817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 345/692] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:33.429455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.455456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.539178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.548958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d5519-199d-4722-b890-e882c94d18a0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.588656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.589313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.589598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.589872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.615634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57b62-e0c5-4363-aa29-4f2a853d5dbe None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.656164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:33.657260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.657549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.657822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.706477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.708015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1bdad8-5fdf-4c50-95d5-4bd36f4e9aea None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.747305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:33.748758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4f719-6ad4-4586-8485-ebad73d4c15b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:33.794329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.828139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:33.841278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 348/693] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:33.842841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e39195-7252-418c-8fa3-30b5630d0294 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.843232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e39195-7252-418c-8fa3-30b5630d0294 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.844155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e39195-7252-418c-8fa3-30b5630d0294 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.850589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 349/694] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:33.850589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e39195-7252-418c-8fa3-30b5630d0294 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:33.866024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 346/695] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 10:38:33.867745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:33.868233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:33.868744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:33.891314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:33.892753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:33.970957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e39195-7252-418c-8fa3-30b5630d0294 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:33.985223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 347/696] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:33.986337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.015546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.017404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.017901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.018430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.033219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.035224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.043968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:34.061658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:34.062697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:34.063844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:34.063844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:34.063844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:34.063844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:34.066570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 350/697] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:34.067908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.117121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11098-0755-4edf-b660-08094ad0159f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.122715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.163178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.164018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.164304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.164574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.175461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.177070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.189047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac713-ff37-4cdc-9657-56eefe21ec0b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:34.195586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:34.196678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:34.197855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:34.197855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:34.197855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:34.197855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:34.201464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 348/698] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:34.202376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.240588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.241689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.242182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.242695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.243501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.245711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:34.266677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:34.267880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:34.268729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:34.272407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 351/699] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:34.273572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.308773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.319237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61e8a5-021f-4971-bc8d-f34afe3cca49 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.355888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.356745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.357119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.357529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.358590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:34.359233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.360499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.373216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 349/700] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:34.374244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.404510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecf45-5b9d-471d-a38b-c0c72a075e8f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.415765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.444759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.445595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.446003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.446402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.447224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:34.447627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.467562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 352/701] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:34.468587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.506090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a9d45-9c55-43af-ac16-7bc39445f9d9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.551582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.552502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.552935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.553387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.554803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:34.555266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.557226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.564315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 350/702] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:34.565363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.585461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fdcdd-33fe-42ad-a02f-8251fe26f7b4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.590030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.624807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.625704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.626102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.626475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.635220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.636868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:34.652935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:34.654121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:34.655235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:34.657291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 353/703] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:34.659833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.689753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.714202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3c75d-d7f7-44da-9bda-b1c6b2de0781 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.749249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.750246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.750623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.750990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.760729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.763164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.782044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93842e11-a590-4c3f-9720-658918e2eb58 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:34.784145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:34.785201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:34.786341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:34.789450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 351/704] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:34.790512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.826360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.827054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.827335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.827609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.828317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:34.828629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.830114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.842305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 354/705] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:34.843093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.883925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.911639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1d478-f26e-4c77-8342-d698dcdd5b72 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:34.949412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:34.950159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.950448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.950726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.951534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:34.951865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:34.953501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7d3d-6986-4682-82f8-28769316a4a8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:34.964856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:34.967097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 352/706] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:34.968568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f22eb-2b25-4d61-a87c-c7b136f8ebc8 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:34.968927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f22eb-2b25-4d61-a87c-c7b136f8ebc8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:34.969168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f22eb-2b25-4d61-a87c-c7b136f8ebc8 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:34.975573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 353/707] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:34.975573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f22eb-2b25-4d61-a87c-c7b136f8ebc8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:34.982665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794f126-d3c5-49b8-9a92-3280ec8027a7 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.021269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.022345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.022849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.023364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.024405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:35.024941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.026641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6233ebc3-8feb-420e-9d16-3baf12a3ecff None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.042082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 355/708] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:35.043667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.044120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.044403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.061798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:35.063039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:35.094361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f22eb-2b25-4d61-a87c-c7b136f8ebc8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.132530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.133304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.133744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.134275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.135138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=ef2bef098cce41279cc9a5d81f0dd782) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:35.135525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.136401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 356/709] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:35.137146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.137258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.149074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:35.153568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 354/710] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:34 2022] GET /identity/v3/role_assignments?user.id=ef2bef098cce41279cc9a5d81f0dd782&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:35.171932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.295586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ab936-850f-4fa5-8835-eccb28a477e7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.319470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:35.345273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.346115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.346408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.346683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.361440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.363225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.365209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07c316-c8c6-466b-b349-4f95b5eca078 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:35.385057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:35.386022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:35.386926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:35.386926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:35.386926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:35.386926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:35.386926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 355/711] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:35.392385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.414872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.416298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.416780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.417268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.418410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:35.419443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.421602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502946d-c4da-4789-9605-40b2b57a8640 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.431117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 357/712] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:35 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:35.455123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:35.510113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2177-2354-496d-b211-7e6cf0c2855b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.519738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9542448-f258-4f37-9c48-347b79fb418b None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.520358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9542448-f258-4f37-9c48-347b79fb418b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.520909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9542448-f258-4f37-9c48-347b79fb418b None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.523504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 358/713] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:35.531913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.532427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.532803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.556338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:35.557987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:35.567185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.567868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.568150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.568425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.569160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:35.569464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.570963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.585511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 356/714] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:35.590746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.653796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 359/715] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:35.677636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.708702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96fea7-9ff6-49f0-8914-cf537f69583d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.722613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:35.745641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.746936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.747218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.748526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.750280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.759639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:35.777015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 357/716] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:38:35 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:38:35.801167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fcb5a-2c3e-41eb-b484-6e2e11771be7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:35.841391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:35.842485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:35.842942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:35.843359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] PATH_INFO: `/v3/services/network` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:35.844174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:35.845702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:35.859277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:35.860613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:35.861678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:35.863603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 360/717] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:35 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:35.870764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:35.989146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5ca20-9537-4e3c-8b28-4015f78cb565 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:36.027513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:36.028588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:36.029074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:36.029551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:36.030576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] RBAC: Adding query filter params (name=network) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:36.031102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:36.032752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d23ae-4535-4950-b871-a5f585a25ccb None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:36.042434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 358/718] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:36.047981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:36.172770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ae7-025c-4aa6-9700-2f88ed9365ad None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:36.213528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:36.214613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:36.215120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:36.215670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:36.216587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] RBAC: Adding query filter params (type=network) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:36.217142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:36.218755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a097414-f3a1-480e-99f4-a616d4bdf4bb None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:36.224591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 361/719] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:37.594070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1929-a5ed-41ec-a6ca-04926023afe6 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:37.594636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1929-a5ed-41ec-a6ca-04926023afe6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:37.594636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc1929-a5ed-41ec-a6ca-04926023afe6 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:37.596229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 359/720] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:37.601908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb0b4-87d6-4c10-8b98-e061840d20bf None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:37.602439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb0b4-87d6-4c10-8b98-e061840d20bf None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:37.602562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cb0b4-87d6-4c10-8b98-e061840d20bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:37.611344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:37.622435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c39cb0b4-87d6-4c10-8b98-e061840d20bf None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:37.624408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c39cb0b4-87d6-4c10-8b98-e061840d20bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:37.713046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 362/721] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:37.747108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc1929-a5ed-41ec-a6ca-04926023afe6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:37.796893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:37.899855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc1929-a5ed-41ec-a6ca-04926023afe6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:37.951527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:37.952205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:37.952498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:37.952779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:37.953514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:37.954920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dadbd0-405e-4253-bc22-46259fcdfc3d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:37.989938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 360/722] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:38:37 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:38.007250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099e12-e7f8-4f65-9d19-d04537b274b6 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.007613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099e12-e7f8-4f65-9d19-d04537b274b6 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.007817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099e12-e7f8-4f65-9d19-d04537b274b6 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.009573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 363/723] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:38.015626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.015926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.016191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.034771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:38.036029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:38.123590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce698d0-2b85-4c24-956d-08781820f463 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.124051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce698d0-2b85-4c24-956d-08781820f463 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.124338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce698d0-2b85-4c24-956d-08781820f463 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.125920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 364/724] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:38.131681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.131969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.132251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.135425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 361/725] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:38.158228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.167029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:38.168534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:38.236447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:38.278045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da140896-f984-4884-b9b4-a9fa78e4c811 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:38.319600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:38.320412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.320698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.320981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.332223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:38.333796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 365/726] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:38.345726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:38.346724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:38.347611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 362/727] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:38.350455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.357816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.428650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:38.459126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:38.472579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479020-47cd-421d-a11c-8feda0bedc7c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:38.492278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446f683-1e4a-4674-b829-bf7ac40b7989 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:38.540646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:38.541316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.541599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.541875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.542655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:38.542958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:38.544341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:38.560333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 366/728] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:38.561530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:38.562359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.562641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.562914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.569063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.586888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:38.599881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:38.601724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:38.602593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 363/729] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:38.608825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.780259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f00194-a20c-4e82-8ba8-b9e382da19e6 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:38.794390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4723-2356-436b-890d-5222b7c40f00 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:38.840485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:38.847944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:38.849327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.849616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.849928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.850850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:38.851168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:38.852600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d35d9d-cbb5-42e4-8802-767f3adb6a7a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:38.858408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:38.859217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.859503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.859774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.860419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:38.866086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 364/730] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:38.867421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb88f4-efa3-466d-b071-058d2310b6b1 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.867713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb88f4-efa3-466d-b071-058d2310b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.867992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb88f4-efa3-466d-b071-058d2310b6b1 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.872955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:38.873534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 365/731] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:38.874354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb88f4-efa3-466d-b071-058d2310b6b1 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:38.874695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-f3d48c7d-bff1-481f-80d7-09af436e140f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:38.897718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:38.898901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:38.900230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:38.900230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:38.900230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:38.900230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:38.902082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 367/732] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:38.903660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:38.903958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:38.904241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:38.928674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:38.930735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:39.008560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb88f4-efa3-466d-b071-058d2310b6b1 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.027292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 368/733] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:39.029029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.057229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.058068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.058358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.058647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.069982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.071484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:39.090034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:39.091010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:39.092263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:39.092263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:39.092263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:39.092263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:39.092263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 366/734] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:39.092617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.157037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.163180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba55b7-b207-41a8-b259-0aefee7af975 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.172566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.201049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.201717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.202031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.202317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.203155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:39.203467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.204955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.206360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3269-4c06-458a-bf27-0b5f63c7a002 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.221561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 369/735] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:39.221561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.246577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.247378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.247659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.247932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.258664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-fd2cb4fe-2f3e-417e-99d0-246416879e32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:39.266484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:39.267402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:39.268835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 367/736] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:39.269647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5282-adcb-4ace-8ffd-731aff08ba44 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.269647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5282-adcb-4ace-8ffd-731aff08ba44 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.269779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5282-adcb-4ace-8ffd-731aff08ba44 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.271740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 368/737] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:39.276187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa5282-adcb-4ace-8ffd-731aff08ba44 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.342845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f1064-59c8-4c8e-bf24-57888cccb880 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.377453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa5282-adcb-4ace-8ffd-731aff08ba44 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.381438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.382344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.382798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.383230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.384095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:39.384700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.386106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.391755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.403666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 370/738] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:39.403666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.413887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.414765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.415061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.415340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.424038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.425514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.432340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-27c18384-ae19-4864-84de-46db63d63c57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:39.441583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:39.443242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 369/739] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.449297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.450217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.462236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:39.463385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:39.514343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc3a-8bb4-4557-8dc2-f7153149f014 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.537889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 370/740] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 10:38:39.549228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.549886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.550195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.550470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.550804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.551798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:39.552105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.553487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.564576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 371/741] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:39.565389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.657110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00fe9f-83f8-457d-a8a4-288df6227159 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.658583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.661984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.673558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99185c5b-e1c6-477e-bf98-6745a6bb75b0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.693550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.694375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.694661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.694939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.701977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.703442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.707766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.708426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.708707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.708984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.709811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:39.710149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.711559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:39.717721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:39.718693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:39.719654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 371/742] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:39.720258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.732786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 372/743] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:39.733791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.827551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296392c-534e-4498-aa35-8dfd919cb205 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.834248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed0593-cfc5-4ccb-a371-4f47b388d57d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:39.864548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.865476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.865981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.866458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.867247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.868690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.868993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:39.869784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:39.870113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:39.870444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:39.881455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:39.890360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 372/744] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:38:39 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:39.890593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:39.890688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.901315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:39.916848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:39.917997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:39.919145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:39.919145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:39.919145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:39.919145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:39.919609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 373/745] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:39.921114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:39.921943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.035354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432eec1b-0818-4688-b131-a3276e7e6575 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.070549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ab383-af69-48cc-be86-003faad846c0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.075271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.076137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.076524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.076949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.078945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:40.078945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.079869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.106606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 373/746] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:40.107355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.124324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.125277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.125494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.125723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.138247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.139722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:40.161033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:40.162189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 374/747] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:40.163292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.202081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.230018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e30545-49d8-441a-bbf3-7f63f07d2335 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.237264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.272574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.273342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.273700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.274094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.275000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:40.275491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.276931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.290574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 374/748] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:40.291527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.295655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3f82e-19a1-4114-8147-da38841b3e79 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.330790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.331461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.331745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.332025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.406530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9863ce-cc64-4c31-8949-1ce4e96fd948 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.406859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.408142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.444754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.445540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.445833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.446144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.446871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:40.447182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.448685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.451201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.462728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.465425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 375/749] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:40.466645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.518920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 375/750] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:38:40.519843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.581673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d885cfa-1085-4b83-bb8f-0851a1648e0e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.618615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.619586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.620195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.620498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.622622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6860ba-a117-4dd4-892e-eb6cf27cc8e3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.627996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.630005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:40.647099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:40.648063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:40.648995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 376/751] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:40.649587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.663290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.664088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.664380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.664658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.671651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.673107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5dbc297c-0e74-43ac-8e16-1ba53783c10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:40.686841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:40.687720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 376/752] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6677-8a99-4c94-9faf-ba278a19c4d7 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.694309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6677-8a99-4c94-9faf-ba278a19c4d7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.694938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6677-8a99-4c94-9faf-ba278a19c4d7 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.694938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 377/753] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:40.694938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb6677-8a99-4c94-9faf-ba278a19c4d7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.711411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.716579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.764264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca35a6d-f8d7-45ef-beec-a2c92ae2b321 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.802856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.803660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.803944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.804254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.805374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb6677-8a99-4c94-9faf-ba278a19c4d7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.814925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.816378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:40.834481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:40.835572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:40.836659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:40.836659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:40.836659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:40.836659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:40.842087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 377/754] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:40.843207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.844870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.845614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.846162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.846508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.848304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:40.848689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.850214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e032335-dbc6-49f7-8a3f-bae95fbd59cc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:40.874190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 378/755] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:40.874190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.874190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.874190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.890726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:40.892024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:40.948413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793185c-81f1-4b51-8d45-9d8a2dd9b83b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:40.952684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.958012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 379/756] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:40.958801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:40.963937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:40.993800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:40.994648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:40.994930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:40.995964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:40.996759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:40.997068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:40.998528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.015921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 378/757] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:41.015921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.076985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0b15-6ac6-4a85-9315-e27d7a562941 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.115198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.115675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd871b7-30db-4474-8502-840518cca738 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.116118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.116501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.116872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.117703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:41.118142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.119659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.132384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 380/758] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:41.138518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.153060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.153713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.154024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.154302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.192016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.203625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.226402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.227871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.255094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37ebe1-c060-4fbb-bcf4-60506984b384 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.291860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.292664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.292950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.293400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.294209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.295642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:41.308900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:41.309864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:41.310778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 381/759] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:41.311645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.334745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 379/760] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:38:41.339996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.406396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.419829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6bef4-7991-4e66-8575-842d2e4e99cf None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.440208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a4f6-bc86-49eb-83a8-a47c420b328f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.446035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.460216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.461352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.462262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.462740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.465725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=8344a4e41ea548568e99784e3a6ef493, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.468211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.476333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.477327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.477685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.478051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.493987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 382/761] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:41 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:41.493987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.494278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.494278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:41.509901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:41.510887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:41.511790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 380/762] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:41.512276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.622008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979117f-80fa-4a50-a8ac-d3cadf0e98f3 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.639516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6991af5-acc7-44ef-8a45-3aecab63cb63 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.650403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.659241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.660176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.661007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.661300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.684870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.685626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.685915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.686227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.686969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:41.687282 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.688742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.694882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 383/763] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:41.695786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.798349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:41.814721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd8b56-33d4-4919-b5db-2e83d3cd9e16 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:41.828750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.830473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.852652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:41.853494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:41.853885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:41.854016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:41.855048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:41.855986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:41.855986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:41.855986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:41.855986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:41.856207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:41.856207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:41.856207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:41.857902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:41.859812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 381/764] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:41.861134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.870656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 384/765] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:41.871570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:41.989245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e5d0f-07a0-49fc-b1de-7240b080fdc5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.005427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71286736-84bc-4930-9ee9-629e3c70aa74 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.016487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.035322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.036373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.036949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.037405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.038700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.039470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.041024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.043734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.044493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.044837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.045372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.045979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.046366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.047807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.049591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 382/766] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.050678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.071961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 385/767] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.072961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.102348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.172905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4b7a0-1e09-41a0-9435-31e323719499 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.186909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30596e-6d07-4eb6-bfdd-4e081b5774ab None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.215453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.216097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.216378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.216738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.217479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=ef2bef098cce41279cc9a5d81f0dd782) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.217785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.219242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.224404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.224900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 383/768] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/role_assignments?user.id=ef2bef098cce41279cc9a5d81f0dd782&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.225500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.225681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.226166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.226445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.227244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] RBAC: Adding query filter params (interface=public, service_id=537e2c62765747e4ba9d7fe7bab32e5f, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.227561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.228982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae2b79e-a084-4cd7-a8a3-5237c40c476f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.245828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 386/769] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/endpoints?service_id=537e2c62765747e4ba9d7fe7bab32e5f&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.247456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d6e5-0ab2-4de5-bc88-9f8da330bb0f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.247895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d6e5-0ab2-4de5-bc88-9f8da330bb0f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.248317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9d6e5-0ab2-4de5-bc88-9f8da330bb0f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.253892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 387/770] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:42.255399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d6e5-0ab2-4de5-bc88-9f8da330bb0f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.293278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.381520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9d6e5-0ab2-4de5-bc88-9f8da330bb0f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.387599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb76f01-6d4a-4785-986a-61bab78dae30 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.402113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.422847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.423721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.424098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.424483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.425585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.427327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d45aed-f2c0-46d8-b5a3-fc53a6b116f5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.434563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 388/771] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.435933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.436226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.436500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.451254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.452367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.452650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.452928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.456157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:42.457515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:42.461641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.463123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:42.493032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:42.494048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:42.494949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 384/772] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:42.504108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.564733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 389/773] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:42.587070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.640818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.693017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218bcc0-2102-49a1-a22a-29e092e85ca9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.732357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.733495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.734055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.734581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.735570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.736085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.737987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.744324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86c4fd-6459-430f-9041-7e24083bf38e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.757347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 385/774] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.763156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.789759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.791465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.791979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.792452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] PATH_INFO: `/v3/services/network` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.793590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.795927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.799168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:42.814914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:42.816126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:42.817192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:42.820227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 390/775] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:42.826412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:42.885103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca2fa4-f69c-462c-b19a-7034d6c5a6af None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.921841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.922602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.922980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.923384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.923779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:42.924656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=086bf8e27f4745c3a01adcfe8d617cf2) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.925046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.927201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.933385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 386/776] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/role_assignments?user.id=086bf8e27f4745c3a01adcfe8d617cf2&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.935252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b7810-f564-4396-bc00-e4d79ba41d06 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:42.972004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:42.973470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:42.973916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:42.974379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:42.975301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] RBAC: Adding query filter params (name=network) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:42.975765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:42.977287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:42.982908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 391/777] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:42.989571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:43.092388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:43.102586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03045e-1685-4289-9f47-b22a5405c138 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:43.140554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:43.141377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:43.141733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:43.142112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:43.142906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] RBAC: Adding query filter params (type=network) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:43.143285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:43.144779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:43.150975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 387/778] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:43.159263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:43.275875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf483bff-9f5e-44ba-9370-b3d8b4c83420 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:43.310285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:43.310946 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:43.311227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:43.311506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:43.312295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] RBAC: Adding query filter params (interface=public, service_id=897a2944e8af4a06a06798bb11760e19, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:43.312609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:43.314011 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb1527-2861-44cc-85c9-3c78912a96be None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:43.320466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 392/779] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:38:43 2022] GET /identity/v3/endpoints?service_id=897a2944e8af4a06a06798bb11760e19&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:43.325787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:43.453754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ead922-5d02-474f-a6f3-89edc518c316 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:43.498812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:43.499404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:43.499683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:43.499955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:43.500633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:43.502043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7791dc-656c-47db-be97-243fcab2fd56 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:43.510585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 388/780] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:38:43 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:43.969814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99a8b-dc7e-4438-b767-5eea7cbbb56b None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:43.970356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99a8b-dc7e-4438-b767-5eea7cbbb56b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:43.970436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf99a8b-dc7e-4438-b767-5eea7cbbb56b None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:43.972061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 393/781] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:43.978241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:43.978681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:43.978782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:43.996558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:43.997753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:44.012265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:44.074097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 389/782] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:44.092436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99a8b-dc7e-4438-b767-5eea7cbbb56b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:44.113145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:44.227283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf99a8b-dc7e-4438-b767-5eea7cbbb56b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:44.263129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:44.263927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.264210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.264484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.277269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:44.285010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:44.285986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:44.287280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 394/783] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:44.293976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:44.427525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac124e62-7014-4c73-9512-47b5894efc5a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:44.478610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:44.479211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.479493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.479772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.480517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:44.480818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:44.482219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:44.493596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 390/784] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:44.499544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:44.630802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:44.656431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a3079-3759-41f2-9912-ce4075929a2f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:44.697509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:44.698337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.698624 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.698902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.713479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:44.715016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-18279435-8e7b-4335-8b5c-0c2458a621eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:44.743227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:44.744153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:44.745488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:44.745488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:44.745488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:44.745488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:44.745488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 395/785] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:44.751704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:44.778893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:44.892711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6d86f-576f-4709-86a5-4a70d4e6bb8a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:44.914746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8686b-415e-42d9-aecf-ad9f1667fcee None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.915031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8686b-415e-42d9-aecf-ad9f1667fcee None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.915321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8686b-415e-42d9-aecf-ad9f1667fcee None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.916927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 396/786] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:44.922129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.922418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.922696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.940140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:44.941347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:44.943718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:44.945161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:44.945444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:44.945718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:44.946469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:44.946774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:44.948133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:44.962568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 391/787] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:44.968195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.068806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 397/788] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:38:45.091195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.103924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.139537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1cd69-65b2-40cb-8bb1-63a0822a566d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.180972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.181826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.182140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.182418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.190675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:45.192719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:45.209501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:45.210527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:45.211881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 392/789] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:38:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:45.216549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.237136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.324192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f7955-1f8d-4420-acf5-ec937fdf3bd8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.370303 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.371654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.372117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.372542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.380285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ceacad-3cc0-4ab6-9cdb-421a4679c7e4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.384431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:45.401286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:45.402462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:45.403311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:45.406200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 398/790] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:45.414127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.414788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.426913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.427615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.427900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.428183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.429042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:45.429355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:45.430841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:45.468594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 393/791] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:45.474642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.586087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d82a16-b18a-4ad9-b36d-f300820fb409 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.603844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1d0c-25c1-42c8-a2f4-ea40d1ff6825 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.616175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.616835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.617123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.617411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.625881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.643313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.644403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.645428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.645927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.646982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:45.647520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:45.649209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60afb54-804f-4776-a194-e9a58f6b5210 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:45.659542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 399/792] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:45.662005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b23fc-0923-49f7-a094-877b3649a372 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.662547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b23fc-0923-49f7-a094-877b3649a372 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.663042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b23fc-0923-49f7-a094-877b3649a372 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.664992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 400/793] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:45.666043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b23fc-0923-49f7-a094-877b3649a372 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.685164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.697979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:45.699541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f57936-3741-49c4-a74a-0b23f3452457 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:45.776525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b23fc-0923-49f7-a094-877b3649a372 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:45.807375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 394/794] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 10:38:45.808574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cf78b-4447-49b7-83cb-5bc4eabaa2b3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.808831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cf78b-4447-49b7-83cb-5bc4eabaa2b3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.809111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cf78b-4447-49b7-83cb-5bc4eabaa2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.813387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:45.814480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.814986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.815474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.826877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-910cf78b-4447-49b7-83cb-5bc4eabaa2b3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:45.828154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-910cf78b-4447-49b7-83cb-5bc4eabaa2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:45.829218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:45.830926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:45.849504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:45.850475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:45.851394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:45.851394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:45.851394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:45.851394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:45.852032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 401/795] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:45.852886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.889080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.901580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 395/796] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:45.903083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6c60-560e-450a-8d7a-e444b32e3916 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:45.903446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6c60-560e-450a-8d7a-e444b32e3916 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:45.903793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8a6c60-560e-450a-8d7a-e444b32e3916 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:45.905283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 396/797] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:45.905893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a6c60-560e-450a-8d7a-e444b32e3916 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:45.929245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:45.968881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255a763-0b73-4697-9d2d-10421cde452a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.005616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.006552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.006835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.007113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.011020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8a6c60-560e-450a-8d7a-e444b32e3916 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.018203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.020197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-4260e10d-cac4-4b82-a256-6ffbbd95e955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:46.039123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:46.040054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:46.040953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:46.040953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:46.040953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:46.040953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:46.040953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 402/798] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:46.041679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.041996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.042269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.050585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.051783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.052068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.052342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.053064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.053370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.054774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.059944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:46.061147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:46.071609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 397/799] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.071609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.147205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 403/800] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:46.148615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.166814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.171255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.193255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585421fd-7503-4ba5-b9ab-6feee1ee772d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.228745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.229559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.229840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.230144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.230803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.232165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:46.246822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:46.247810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:46.248738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 398/801] 10.222.0.36 () {56 vars in 1129 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:46.249617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.269288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17750997-2046-4acd-96fa-88992a7c8253 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.304071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.304736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.305025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.305304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.306054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.306367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.307752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.319462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 404/802] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.320265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.356375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de3e45-7c5f-47e3-9568-c5255f22fb5d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.389861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.390923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.391203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.391474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] PATH_INFO: `/v3/users/swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.394842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.398893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.400272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:46.414273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:46.415425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 399/803] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:46.416375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.420497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.423944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447ba6f-10b4-4cb9-ac6b-83d96c2a34d8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.458902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.459811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.460106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.460388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.470390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5a66ce6d-d9cf-4741-831c-39a89a17af06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:46.478686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:46.479840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:46.480874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:46.484654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 405/804] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:46.485890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba667a-e6fe-4a57-87a7-772a77e02416 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.486206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba667a-e6fe-4a57-87a7-772a77e02416 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.486492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba667a-e6fe-4a57-87a7-772a77e02416 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.487775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 406/805] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:46.489144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba667a-e6fe-4a57-87a7-772a77e02416 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.517003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1108bcec-15ec-4800-abf9-fbf92904edda None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.549538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.550213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.550500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.550771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.552458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.552873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.554260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.558932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 400/806] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.559697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.590172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba667a-e6fe-4a57-87a7-772a77e02416 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.622834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.623603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.623891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.624156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.626800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.627733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.631532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.633069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:46.647231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:46.648113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:46.649047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 407/807] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:46.649541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.665591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a377b9-dacc-4809-b6c3-f060dc347dde None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.699251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.699925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.700208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.700486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.701326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.701638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.703099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c17fe-7703-4068-a53e-d2471915b1b1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.719878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 401/808] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.721104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.721394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.721675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.737839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:46.739618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:46.753255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2f25-edd2-49ad-9789-d152452a1f1c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.788204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.788870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.789154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.789429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.790197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.790509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.791884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.799894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 408/809] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.800681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.805207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 402/810] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:46.806067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.862361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.870766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:46.906884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125f60e-b62e-45ff-9777-b810901203ad None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.913003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b591c1-82c1-47e4-abd2-639969917f12 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:46.939500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.940165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.940452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.940726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.941544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.942383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.943812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.947339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:46.947995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:46.948275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:46.948550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:46.949247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:46.949553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:46.950932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:46.955183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 403/811] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.960320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 409/812] 10.222.0.36 () {56 vars in 1124 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:46.960450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:46.960546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.057567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c886202-a800-4e57-bab4-8c84ddf99e0a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.058734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ddaf1-31db-4cfa-8c6b-6163e2fb0c88 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.074591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.080118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.091446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.092234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.092525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.092803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.093375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.094189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.094471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.094746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.103306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.104759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.105031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.106494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:47.121938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:47.122818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:47.123701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:47.123701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:47.123701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:47.123701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:47.124261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:47.125088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:47.126025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:47.126025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:47.126025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:47.126025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:47.126025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 404/813] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:47.126616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.126890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 410/814] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:47.127982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.225236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225dc1-a214-46a5-bc22-7f852c08d25d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.226344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c40b06-e92a-43b7-9533-95f34c7fd1e5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.259386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.259588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.260053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.260340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.260434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.260619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.260720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.260908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] PATH_INFO: `/v3/services/network` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.261450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:47.261551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.261768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.262948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.263240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.269399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 411/815] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:47.270213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:47.274851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:47.275913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:47.277335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 405/816] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/services/network => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:47.278204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.316220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.323372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.383275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fabb08-75d4-4b1d-9c5e-9e9d6590f34d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.403535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c34c0-95d4-424d-aac7-960897b1201b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.419663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.420332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.420615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.420889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.421636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.423073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.445022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.445530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 412/817] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:38:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:47.446227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.446401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.446820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.447099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.447870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:47.448184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.449601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.465854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 406/818] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:47.468887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.595894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.599784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.614107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862a48b-8bba-4548-9a39-be0a79494c70 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.615725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811891c-2366-4e85-bbc7-44f3d67b7287 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.659528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.660764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.661037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.661311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.662442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] RBAC: Adding query filter params (name=network) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:47.662442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.663740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.668383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.668633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 407/819] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:47.669431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.669701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.670307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.670792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.671714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:47.672226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.674093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.689037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 413/820] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:47.691311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.796977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64189573-6149-468d-abb3-91b8001d3efb None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.798056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce6e2c-8711-4dc6-84fa-f71dcdb0f916 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.833249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.834105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.834266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.834400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.834653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.835150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.835437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.835725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:47.836434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] RBAC: Adding query filter params (type=network) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:47.836740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.838161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.854054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 414/821] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:47.854054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.854297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:47.854297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:47.860833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:47.869911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:47.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 408/822] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:47.872442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:47.891000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:47.955177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e1ee0-44f7-465f-9beb-54e9c62f667a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.978797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87060303-23b2-4c32-860b-c70aa7e29a08 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:47.992008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:47.993112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:47.993544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:47.994007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.001284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.002966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.016747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.017413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.017699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.017990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:48.018703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:48.020378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:48.021487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:48.022086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.022086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.024051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 415/823] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:48.025089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.050034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 409/824] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:38:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:48.051007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.204963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c46ba-3c44-486c-81e6-b9700c2111a8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.214336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:48.243082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:48.257320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.258032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.258501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.259157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.260035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:48.260450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.262706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.284971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 416/825] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:48.290410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.330787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47096146-5009-4d29-8623-b2b52a1cbdd4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.372729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.373403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.373690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.374017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.374824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:48.375137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.376595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.393157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 410/826] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:48.398965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.412224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f7f00-8511-4332-b646-10aa2e0afc9c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.447119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:48.456641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.457711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.457851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.458082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.471057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.472763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:48.492462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:48.493577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:48.494539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:48.494539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:48.494539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:48.494539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:48.494539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 417/827] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:48.500068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.522803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c60e4-8494-48cf-9fab-fb9a6234e370 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.568955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:48.577814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.578777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.579064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.579341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/ef2bef098cce41279cc9a5d81f0dd782/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.583817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=ef2bef098cce41279cc9a5d81f0dd782, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.586077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.610290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 411/828] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:48 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/ef2bef098cce41279cc9a5d81f0dd782/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:38:48.643664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4fa4-9264-46e6-80d1-ceaca1683461 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.685320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.686098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.686387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.686665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.687424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:48.687734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.689157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.703158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 418/829] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:48.708571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.820665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dc998-33a9-473b-bc16-179ede8b8177 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:48.856646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:48.857456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:48.857740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:48.858043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:48.865800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:48.867310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:48.881338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:48.882322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:48.883214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 412/830] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:48.888465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:48.933199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:48.993925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f73ecd-b356-43ca-8671-0f81853ae6a3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:49.028442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:49.029107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.029393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.029668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.030488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:49.030800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:49.032233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77234eb7-53de-43e1-ab31-25e8bd10ad3a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:49.045326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 419/831] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:49.051385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:49.109070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:49.166777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec356b81-c15c-43fc-b7d7-739d70599e4a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:49.204718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:49.205617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.205902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.206207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/086bf8e27f4745c3a01adcfe8d617cf2/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.221974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=086bf8e27f4745c3a01adcfe8d617cf2, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:49.224345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98af3ad-385a-4784-988b-c36f6aa92400 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:49.251350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 413/832] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:38:49 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/086bf8e27f4745c3a01adcfe8d617cf2/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:38:49.682135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7936-5b89-4092-8ff5-f61c5a95ef1b None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.682853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7936-5b89-4092-8ff5-f61c5a95ef1b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.682853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2db7936-5b89-4092-8ff5-f61c5a95ef1b None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.684384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 420/833] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:49.690504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230db3-6cb6-4ab2-817c-cf1ac28c9a64 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.690792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230db3-6cb6-4ab2-817c-cf1ac28c9a64 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.691065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230db3-6cb6-4ab2-817c-cf1ac28c9a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.709023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-03230db3-6cb6-4ab2-817c-cf1ac28c9a64 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:49.710260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-03230db3-6cb6-4ab2-817c-cf1ac28c9a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:49.780245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 414/834] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:49.802252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db7936-5b89-4092-8ff5-f61c5a95ef1b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:49.876474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da235-e71a-4a6d-bdda-8379b973a27d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.876778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da235-e71a-4a6d-bdda-8379b973a27d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.876988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48da235-e71a-4a6d-bdda-8379b973a27d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.878664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 415/835] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:49.884247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.884539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.884821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.902070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:49.903274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:49.910357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:49.913498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2db7936-5b89-4092-8ff5-f61c5a95ef1b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:49.934433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:49.955732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:49.956550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:49.956836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:49.957247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:49.957908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:49.959423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:49.972848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:49.974819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:49.976349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 421/836] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 23 10:38:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:49.983078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:49.983514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 416/837] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:38:50.006774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:50.107371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8291559-4bfb-43a1-81c5-770e980f4c7b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:50.107673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338dce-32a6-412a-82eb-1f00ab7b1560 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:50.143505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:50.144269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.144676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.144960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.145756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:50.146093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:50.146214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:50.147111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.147602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:50.147841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.148216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.149139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:50.151560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:50.152843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 422/838] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:50.158058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:50.164728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:50.166033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:50.167103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:50.171830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 417/839] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:50.172772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:50.194377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:50.212770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:50.285741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98a1d4-035a-4a7f-9414-9de320acfe65 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:50.296293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0304b-79e7-4a8f-8fde-f6d4308be33c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:50.329631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:50.330351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.330639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.330922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.331638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:50.331977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:50.333372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:50.333675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:50.334602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.335085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.335574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.336477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:50.336982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:50.345981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 423/840] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:50.346184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bad1f6-b1b4-4b3f-9e10-073bb222bf4f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:50.348867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 418/841] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:50.354674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:50.460574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:50.470175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da0b32-839d-4cf2-a278-40272f3e917f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:50.511034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:50.511718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.512005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.512285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.513014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:50.513330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:50.514789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419c8bf-7ce0-4f02-a751-f84e13155a2b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:50.520968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 424/842] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:50.963534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d0a9-7f23-4060-8ed7-9f1149e4bd25 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.963932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d0a9-7f23-4060-8ed7-9f1149e4bd25 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.964109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33d0a9-7f23-4060-8ed7-9f1149e4bd25 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.965696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 419/843] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:50.970986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:50.971449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:50.971526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:50.989095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:50.990324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:51.063070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 425/844] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:51.087093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d0a9-7f23-4060-8ed7-9f1149e4bd25 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.187978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:51.199343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33d0a9-7f23-4060-8ed7-9f1149e4bd25 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.237331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.237331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.237331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.237331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.249983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:51.255583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:51.256533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:51.257905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 420/845] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:51.262909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.356747 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:51.374162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e819c1a2-2be1-4706-bbad-c60407dffcf0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.409888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.410620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.410906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.411188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.411937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:51.412249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:51.413639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed80ddff-fd2b-4ff0-8a71-f9d733b63bd5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:51.422469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 426/846] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:51.428619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.522924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b01f7-ce93-48ed-a738-8ec61169edb5 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.523121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b01f7-ce93-48ed-a738-8ec61169edb5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.523515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b01f7-ce93-48ed-a738-8ec61169edb5 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.525382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 427/847] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:51.530693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.531214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.531693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.532433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51519187-f203-4543-80aa-63173bad5a49 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.549997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:51.551560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:51.571996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.572793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.573079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.573357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.584593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:51.586077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:51.604823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:51.605745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:51.606675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:51.606675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:51.606675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:51.606675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:51.610709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 421/848] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:51.611960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.615422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:51.628534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 428/849] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:38:51.651426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.721118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5cc51-242d-4abc-9bc7-1f2874fa0c65 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.741557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:51.759471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.760163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.760450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.760731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.761470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:51.761779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:51.763229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:51.775507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75d971-ba0e-4e99-82cb-ecf2e84dc59f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.782266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 422/850] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:51.783626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.819556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.821118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.821413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.821696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.835718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:51.846133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:51.847066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:38:51.848432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:51.849060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 429/851] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:51.856976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:51.858815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:51.909504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03af7e-dd17-46fc-bc1a-b494c1b4759c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.958159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:51.958969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:51.959254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:51.959531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] PATH_INFO: `/v3/users/swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:51.969806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:51.971293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:51.972131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182538f-6419-459e-87bf-18366a9b5a1f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:51.991215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:51.992192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 10:38:51.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:51.997250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 423/852] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:51.998991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.017751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.018952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.019497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.019968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.021090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:52.021673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.023283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.030436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.041325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 430/853] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:52.042461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.123634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986509d1-b4ad-49bf-a145-f1dda9b38e4a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.147680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.162606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66de748-6485-426f-98b8-8ccdd15cb5e4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.163425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.164338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.164841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.165353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.166477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:52.166998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.168633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.190298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 424/854] 10.222.0.36 () {56 vars in 1135 bytes} [Mon May 23 10:38:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:52.193095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.203281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.204086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.204379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.204657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.218959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.220629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:52.253606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:52.254596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:52.255467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:52.255467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:52.255467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:52.255467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:52.258033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 431/855] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:52.261975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.320771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.331743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc0453-9c62-4e16-92b1-cdc9a01c0268 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.380595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.381247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.381526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.381804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.382534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b969b1-648a-4182-801b-f93c48952121 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.422743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.423472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.423939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.424221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.425253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:52.425407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.426866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.445301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 432/856] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:52.447793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.457916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.479593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.481106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.578496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947209b-e98d-4990-8238-ceeb152f5607 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.593445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.597037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 425/857] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 10:38:52.603250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.621575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.622729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.623225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.624219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.635795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.638664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-8d209078-a5e1-4dbd-8986-35910cf9de72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:52.657444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:52.658427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:52.659378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 433/858] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:52.660658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ade3-b5b4-4321-8495-6f4edc714b5e None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.660658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ade3-b5b4-4321-8495-6f4edc714b5e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.660658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037ade3-b5b4-4321-8495-6f4edc714b5e None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.662008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 434/859] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:52.666468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037ade3-b5b4-4321-8495-6f4edc714b5e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.695429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.717709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd723b-55e1-4bfe-87f4-a4a6887132e0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.760767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.761563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.761846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.762149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.776840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.778899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.788016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037ade3-b5b4-4321-8495-6f4edc714b5e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-797203f7-0392-491e-80ad-f3a6ac78200e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:52.797055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:52.798270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:52.799200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:52.799200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:52.799200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:52.799200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:52.799200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 426/860] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:52.800061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.800061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.800061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.817937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:52.819199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:52.829028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.829721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.830040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.830322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.831163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:52.831475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.833924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:52.856690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 435/861] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:52.856690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.886737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.902221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 427/862] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:52.903077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:52.955038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b57ac33-a509-4f4e-a2be-dfdb570606e5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:52.971453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:52.988666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:52.989374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:52.989660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:52.989934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:52.990674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:52.990978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:52.992341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7a3a0-5826-459b-bf61-70a8be0d4a3a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.003367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 436/863] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:53.005161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22c40-b0f8-4a09-8582-cf178240cf9d None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.005450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22c40-b0f8-4a09-8582-cf178240cf9d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.005724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22c40-b0f8-4a09-8582-cf178240cf9d None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.006982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 437/864] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:53.007658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22c40-b0f8-4a09-8582-cf178240cf9d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.013836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f673cb4-8a11-44e9-9014-73a4d4a8d785 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.047046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.047696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.047985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.048270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.095966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.107026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22c40-b0f8-4a09-8582-cf178240cf9d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.114369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.115815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30c3b1-cea3-40a3-903c-1526b716b761 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.140712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.141372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.141656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.141936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.142685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.144044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.158342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 438/865] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:38:53 2022] POST /identity/v3/services => generated 269 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:53.159315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.179363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.223933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 428/866] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:38:53.225448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.225813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.226196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.249168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:53.250749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:53.299081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c855d7-439f-4f0b-9e30-54bd0f275c5b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.335672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 429/867] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:53 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:53.335672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.353494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.354596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.355321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.356212 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.366122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.367594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:53.383429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:53.384362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:53.385301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 439/868] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:53.385795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.395020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.443223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.453107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d7d65-8f38-4eef-a14c-eabb6ef093ff None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.490471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.491256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.491558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.491831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.491993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32cdb0-eaf2-40d1-b00d-26d5a0e0a1f5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.502947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.504738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:53.522517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:53.523468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:53.524386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:53.524386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:53.524386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:53.524386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:53.524386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 430/869] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:53.524678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.533129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.534338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.534628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.534909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.536603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.538321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.553932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 440/870] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:38:53 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:53.555014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.646745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.648605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9916-724a-4e1c-9754-86808d0d9d14 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.698216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5836c7-021c-4527-b9f8-3475cbf22cb7 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.698789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.699824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.700316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.700891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.701791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:53.702225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.703785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.714924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.719076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 431/871] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:53.725404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.740608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.741746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.742336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.742917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.743892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:53.744396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.746247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.760545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 441/872] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:53.766178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:53.852298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db9c1d-651e-4a65-8779-0129f6576a96 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.889694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.890423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08bda9-6a9b-4681-8615-d3b4bb2a75a1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:53.891220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.891813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.892279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.893380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=ef2bef098cce41279cc9a5d81f0dd782) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:53.893888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.896099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.903357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 432/873] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/role_assignments?user.id=ef2bef098cce41279cc9a5d81f0dd782&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:53.920542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:53.930910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:53.931720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:53.932002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:53.932276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:53.940312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:53.942417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:53.957999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:53.958945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:53.959817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 442/874] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:53.965378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:54.042722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:54.086434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d5b34-6d95-4f2f-a888-8beb195511e5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:54.124751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:54.125413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:54.125696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:54.126005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:54.126796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:54.127103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:54.128535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadba9c-d570-4120-951e-646dc0178de3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:54.142739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 433/875] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:54.149021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:54.263726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650db35-7b72-4dfb-b41c-fc1bc08c4082 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:54.303079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:54.303731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:54.304013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:54.304283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:54.305104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=086bf8e27f4745c3a01adcfe8d617cf2) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:54.305413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:54.306882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6eefce-7bb9-40ec-9be8-794df2c47ca6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:54.313290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 443/876] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:54 2022] GET /identity/v3/role_assignments?user.id=086bf8e27f4745c3a01adcfe8d617cf2&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:55.460704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2190-4b38-4040-932a-b2a49487c091 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.461250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2190-4b38-4040-932a-b2a49487c091 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.461250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2190-4b38-4040-932a-b2a49487c091 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.462916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 434/877] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:55.468358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.468762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.468937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.486569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:55.487819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:55.538097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:55.566061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 444/878] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:55.582771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2190-4b38-4040-932a-b2a49487c091 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:55.694581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2190-4b38-4040-932a-b2a49487c091 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:55.722914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:55.734087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:55.734908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.735351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.735513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.736141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:55.737510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-efcd1b9e-fb2e-4c75-ae06-c2905ef22280 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:55.750528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:55.752758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:55.755403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 435/879] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 23 10:38:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:55.762042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:55.857664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e28500-b12a-4a91-9807-978e1c34d680 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:55.896689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662f8a-ead1-458e-a213-ee3c5ecfc5eb None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.897026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:55.897158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662f8a-ead1-458e-a213-ee3c5ecfc5eb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.897242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87662f8a-ead1-458e-a213-ee3c5ecfc5eb None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.897628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.897909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.898211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.898905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 436/880] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:55.899025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:55.899209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:55.900580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:55.903540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:55.903834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:55.904116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:55.905782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 445/881] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:55.911253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:55.922727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:55.924180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:55.961780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.014230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 437/882] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:56.032368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.033486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9f14d-898f-45bf-b361-f49c3cbefdd4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.084194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.085665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.086461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.086960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.088050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.088842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.090481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.102450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 446/883] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:56.104313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.146325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7d33-ddfa-41a0-a0d4-2caa600cbdf2 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.166463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.186716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.187871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.188087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.188460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.189224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.190894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:56.203459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:56.204742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:56.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.212775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 438/884] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:56.213820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.242837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94c45e-b64b-4e53-9151-62f92bc715b1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.255430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.287945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.288883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.289174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.289453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.290280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] RBAC: Adding query filter params (interface=public, service_id=d250fd9fd3a14efbb9da293cf850b08a, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.290836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.292834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e02c9-dc37-4c2b-bb97-ebf75ec301d6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.298684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 447/885] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/endpoints?service_id=d250fd9fd3a14efbb9da293cf850b08a&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:56.300016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc72f1-9af4-48e7-96c2-4683730e38c1 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.300312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc72f1-9af4-48e7-96c2-4683730e38c1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.301145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc72f1-9af4-48e7-96c2-4683730e38c1 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.302531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 448/886] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:56.307153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc72f1-9af4-48e7-96c2-4683730e38c1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.329217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a9c7b-8f4e-41d3-8165-ec44c2ccd4df None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.364949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.365607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.365887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.366187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.366888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.367191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.368550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619ec77-c2d7-4114-a19c-8fac49d66a1c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.373759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 439/887] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:38:56.374967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.375253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.375524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.392542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:56.393719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:56.419479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc72f1-9af4-48e7-96c2-4683730e38c1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.442226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.464859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.466208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.466507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.466925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.467673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.469056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.485998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 449/888] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:38:56.485998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.486325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 440/889] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:56.494352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.540755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.612880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce3658-2dc3-45b7-b098-a6163cf8d5eb None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.630089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed0966-ad65-4069-b597-d11e819c67a9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.651116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.651938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.652218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.652629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.653300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.654745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.667583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.668286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6b2fb5d3-c8dd-4700-a246-894ffe13ae91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:56.668389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:56.669331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:38:56.670623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:56.671427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.671427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.671427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.671624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 441/890] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:56.671624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30356ea-ae47-4572-928a-2f5e66b4617d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.671624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30356ea-ae47-4572-928a-2f5e66b4617d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.671826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30356ea-ae47-4572-928a-2f5e66b4617d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.673414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 442/891] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:56.674909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.676434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e5b39-8d1e-4a32-b806-5608e896bee0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.678213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30356ea-ae47-4572-928a-2f5e66b4617d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.682389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 450/892] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:56.683665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.683954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.685003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.703007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:56.704191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:56.731777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.782737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 451/893] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:56.784436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.800210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30356ea-ae47-4572-928a-2f5e66b4617d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.822545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:56.841779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.842784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.843127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.843408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.844270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.844895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.846290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.852246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 443/894] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:56.857552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.900349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd71084-b0b2-4a00-a9ae-e300a7d39fca None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.934617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:56.935194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:56.935474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:56.935747 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:56.936528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] RBAC: Adding query filter params (interface=public, service_id=636f1ccd61b046c6bf55bdd3ce61264e, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:56.936837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:56.938231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:56.943878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 452/895] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/endpoints?service_id=636f1ccd61b046c6bf55bdd3ce61264e&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:56.945038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:56.968662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f22e5-080c-459f-be29-52817f419d59 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:56.972349 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.010142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.011209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.011493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.011766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.023737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:38:57.031867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:57.033050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:57.033973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 444/896] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:57.034745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.054138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.062589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2afe80-8f99-4669-8ce3-ffd757f8438d None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.100424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.101088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.101372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.101648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.102388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:57.102695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.104073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.108866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 453/897] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:38:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:57.110905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.140966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605651bb-028a-428f-9abe-a39b68e8ea4a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.183369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.184441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.184997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.185484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.186975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.190052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.199379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 445/898] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:38:57.246661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fac5b4-5830-48a2-b61b-d8d43eae8638 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.288612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.289797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.290353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.290830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.291828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:57.292355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.294070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.304971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 454/899] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:57.310639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.346602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.430361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92934f-212e-4c35-a3ab-e9f80c1f9c78 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.466116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.467254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.467749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.468241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.479637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.481377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:57.499702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:57.501151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:57.502091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:57.502091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:57.502091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:57.502091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:57.504984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 446/900] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:57.510302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.533795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.617531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a644a5-1ad3-4722-a30a-7b85c2e65af0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.657486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.658192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.658482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.658760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.659494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:57.660326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.661730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.674263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 455/901] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:57.680036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.774142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.793071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0941ea6-a1ed-4a80-a41c-f29bedd92758 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:57.831011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:57.832696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:57.833124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:57.833542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:57.841413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:57.843077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:38:57.857733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:57.858908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:38:57.860088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:57.862532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 447/902] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:57.868191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:57.952260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:57.974531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4efcac-4653-4ff1-81a9-07665b1d6b30 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:58.009681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:58.010616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.011062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.011486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.012481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:58.012931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:58.014559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:58.033704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 456/903] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:38:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:58.043043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.145646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c4dca-c931-40ca-b73c-17764345770c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:58.179090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:58.179823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.180113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.180385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.202765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:58.248624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:58.250115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:58.351785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 448/904] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:38:58.357150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.462213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54347d-3c65-4ae0-94c2-de9dec6de463 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:58.496755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:58.497600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.497883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.498183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.505410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:58.510639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:58.511941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:38:58.531594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:58.532546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:58.533431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:38:58.533431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:38:58.533431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:38:58.533431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:58.533431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 457/905] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:58.538632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.584597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:58.642871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d163ef5-c20a-4267-a0f8-337e69d0be7b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:58.676640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:58.677300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.677581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.677861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.678601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:58.678900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:58.680254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2f9b1-91c3-4370-8610-527223fca7d9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:58.692335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 449/906] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:58.697734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.749227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257460-4c1b-427f-b8c1-300ce75fab10 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.749512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257460-4c1b-427f-b8c1-300ce75fab10 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.749791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3257460-4c1b-427f-b8c1-300ce75fab10 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.751708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 450/907] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:38:58.759847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.760132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.760408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.780233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:58.781411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:58.815679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58a456-01b5-48da-bb00-93295262b414 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:58.861978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 451/908] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:38:58.862171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:58.862171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:58.862171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:58.862171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:58.869152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:58.870716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:58.883258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:38:58.886901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:58.887784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:38:58.888682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:38:58.891508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 458/909] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:38:58.899363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:58.922236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:58.962917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.011047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356c67c-17f1-40ab-9142-a7c030b12e4b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.020662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753f6f5-f03f-4bf5-b4ae-f3096a65c76f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.048344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.049716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.050236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.051264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.052135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.053887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.058439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.059108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.059397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.059679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.060472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:59.060786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.062260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:59.069238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:59.070211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 10:38:59.071057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.074003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 452/910] 10.222.0.36 () {56 vars in 1132 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:59.079866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 459/911] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:38:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:59.084649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.085814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.186375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146069-a959-49d9-bdb1-fd31b8f039f4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.198103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c395aa5-7b7d-458a-a75f-33fe515be194 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.209986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.234225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.235245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.235751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.236221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.237123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:59.237638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.239265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.243648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.244415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.244772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.245121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.246069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] RBAC: Adding query filter params (role.id=75434acb5a4747e3b9e2654f91e27d2e, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=086bf8e27f4745c3a01adcfe8d617cf2) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:59.246443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 453/912] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:59.247523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.249050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.255474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.266055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 460/913] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:38:59 2022] GET /identity/v3/role_assignments?user.id=086bf8e27f4745c3a01adcfe8d617cf2&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=75434acb5a4747e3b9e2654f91e27d2e => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:59.385336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43609b-a0d4-4494-80e5-386f526877da None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.425191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.426552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.426841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.427119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.427845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:38:59.428160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.429561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d8523-7ec8-44a9-8df5-2a1344a424c6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.440314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 454/914] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:38:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:38:59.441743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.496567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7420-156a-4be3-a81a-365eb1e88137 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.496890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7420-156a-4be3-a81a-365eb1e88137 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.497122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7420-156a-4be3-a81a-365eb1e88137 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.498785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 455/915] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:59.503812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed944-108c-4633-9f0c-736ec74e836e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.504101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed944-108c-4633-9f0c-736ec74e836e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.504384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed944-108c-4633-9f0c-736ec74e836e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.522502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6b1ed944-108c-4633-9f0c-736ec74e836e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:59.523704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6b1ed944-108c-4633-9f0c-736ec74e836e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:59.541437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.552078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fbee8-17c1-432d-89bf-1ba67468f569 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.589240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.589896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.590205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.590491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.591239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.592625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.595193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 456/916] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:38:59.603765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199dab-a49d-44b6-baed-21038a359b16 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.604109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199dab-a49d-44b6-baed-21038a359b16 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.604446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199dab-a49d-44b6-baed-21038a359b16 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.606174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 457/917] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:38:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:38:59.611886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.612168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.612399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.622232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 461/918] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:38:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:59.623235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.631662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:38:59.633532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:38:59.658635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.730499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 458/919] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:38:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:38:59.739272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3185-8a91-4547-9b2d-13b4162fb67e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.748718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:38:59.782262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.783238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.783733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.784252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.785711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.787350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.808513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 462/920] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:38:59 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:38:59.848899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:38:59.876322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fece357-ff1c-4edb-bd72-51ff5adb013a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:38:59.923396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:38:59.924317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:38:59.924609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:38:59.924885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:38:59.925587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:38:59.927051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:38:59.940112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:38:59.941763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:38:59.942702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 459/921] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:38:59.950344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:00.050139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:00.061020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b0888-d118-4da1-8d56-bdfee4f9601b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:00.099693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:00.100362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:00.100649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:00.100930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:00.101642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:00.101972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:00.103366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:00.108728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 463/922] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:38:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:00.114399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:00.218501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a6dcb-55fc-4629-83d9-0298f6d3f536 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:00.260738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:00.261759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:00.262381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:00.262867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:00.263791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:00.264304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:00.266172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543e841-614a-433e-8363-49b982a33d83 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:00.272411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 460/923] 10.222.0.36 () {56 vars in 1114 bytes} [Mon May 23 10:39:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:00.278003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:00.384922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a11fd-249b-4ea7-b9f6-543cdbd11037 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:00.422394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:00.423055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:00.423418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:00.423665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:00.424407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:00.426488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42357b5c-e57b-4aca-867a-060585c545dc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:00.449036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 464/924] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:39:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:01.506457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4fceb-3dba-4bf7-9a7f-77c0f07465f1 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.507370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4fceb-3dba-4bf7-9a7f-77c0f07465f1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.507803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4fceb-3dba-4bf7-9a7f-77c0f07465f1 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.509577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 461/925] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:01.515622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.516490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.516981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.539521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:01.540783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:01.543783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:01.625449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 465/926] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:01.648904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4fceb-3dba-4bf7-9a7f-77c0f07465f1 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:01.651982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:01.759991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4fceb-3dba-4bf7-9a7f-77c0f07465f1 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:01.799112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:01.800859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.801388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.801874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.814744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-f04712f9-ff73-4f91-b60d-3f06d22cb95d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:01.823838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:01.824782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:01.825611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:01.833023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 462/927] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:39:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:01.834685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:01.943387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67d436-b4fd-495b-bb13-1525f04ee7a2 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:01.971196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9584a1-2688-4b92-8b28-bcc169a9a929 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.971501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9584a1-2688-4b92-8b28-bcc169a9a929 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.971752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9584a1-2688-4b92-8b28-bcc169a9a929 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.973389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 463/928] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:01.978733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.979021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.979298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.980166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:01.980840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:01.981124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:01.981443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:01.982424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:01.982738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:01.984121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:01.986729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:01.994852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 466/929] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.000253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:02.001436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:02.002991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.067656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.079639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 464/930] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:02.102652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.124915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f34952-dfe7-43bb-95f5-9f8c364ed637 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.162472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.163268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.163555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.163838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.174594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.176648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-49cba60a-7a6f-4e8f-b7cd-6252c50d118b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:02.193963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:02.194892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:02.195794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:02.195794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:02.195794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:02.195794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:02.199690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 467/931] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:02.201205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78679-4d02-4782-8889-e2933078e27d None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.201502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78679-4d02-4782-8889-e2933078e27d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.201785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad78679-4d02-4782-8889-e2933078e27d None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.203228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 468/932] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:02.203855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad78679-4d02-4782-8889-e2933078e27d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.211831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92252b8-07e3-4eed-b213-45937bffcbab None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.252234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.253049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.253337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.253612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.254312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.256114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.266255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-9cf74d35-a49b-428e-8cb0-831274e09341 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:02.268917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:02.270104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:02.270953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 465/933] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:02.271750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.271750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.272877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.289173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:02.290402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:02.330027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad78679-4d02-4782-8889-e2933078e27d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.343670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.363725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 466/934] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:02.365020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.378240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.378937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.379316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.379591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.380312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:02.380619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.382265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.396779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 469/935] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.397690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.481921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc68e-4232-4695-98eb-33760931afa3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.500987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930edf8-2836-4335-a4d5-7a9ce52a5ded None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.512267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.519276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.519962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.520249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.520521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.521253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:02.521558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.523013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.528775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 467/936] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.529440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.546051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.546886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.547176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.547459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.548133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.549564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:02.562709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:02.563750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:02.564580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 470/937] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:02.565380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.591877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.645599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187136f-bfcc-4f43-846e-b648702c0e23 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.678072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2534dd-7875-4532-9a9c-456023223f92 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.680971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.681134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.681473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.681752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.689105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.690585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:02.705257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:02.706258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 468/938] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:02.707767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.719323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.719990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.720284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.720566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.721275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:02.721586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.723558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.737982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 471/939] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.737982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.781899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.820445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91da01-8af5-42f1-bba7-d436c975a220 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.834738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:02.854383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da4857-4a34-4963-9a9b-9748821cdd83 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:02.865576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.866564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.866856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.867134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.867875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:02.868178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.869609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d425c-650e-445a-8cef-5111eda4e25e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.879627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 469/940] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.879627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ffae-da96-42f0-a91c-5d1c091c0a1c None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.879627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ffae-da96-42f0-a91c-5d1c091c0a1c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.879627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2ffae-da96-42f0-a91c-5d1c091c0a1c None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.879627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 470/941] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:02.881291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2ffae-da96-42f0-a91c-5d1c091c0a1c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:02.901362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:02.902377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.902377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.902642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.903467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:02.903785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:02.905258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4847d4-3727-4624-bc59-e9c6d20be1a6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:02.926192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 472/942] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:02.927482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:02.927775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:02.928056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:02.945020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:02.946275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:03.003884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2ffae-da96-42f0-a91c-5d1c091c0a1c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.021982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 473/943] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:03.021982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.043911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.054620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.055382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.055765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.056051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.056776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:03.057088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.058499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.063896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 471/944] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:03.064667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.107948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.132644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a733082-0cd7-4b2f-86f7-e59f5ebc4ef8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.167279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.168075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.168363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.168640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.174146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7511ed-4403-495f-ad0f-c2667ebf2675 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.179262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.180742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:03.200616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:03.201613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:03.202571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:03.202571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:03.202571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:03.202571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:03.207136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 474/945] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:03.207923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.213188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.214105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.214632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.215088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.215972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.217713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.219473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING keystone.server.flask.common [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 10:39:03.245452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.sql.core [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 10:39:03.245452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 10:39:03.245452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [parameters: {'id': '9a9cf6af949049f49d350ddc9c07c129', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 10:39:03.245452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91697) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a9cf6af949049f49d350ddc9c07c129.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a9cf6af949049f49d350ddc9c07c129. May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application conn.query(q) May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:39:03.262433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application result.read() May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 10:39:03.263338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application next(self.gen) May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application yield resource May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application next(self.gen) May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self.session.rollback() May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self.force_reraise() May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise self.value May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application session.commit() May 23 10:39:03.264332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._prepare_impl() May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self.session.flush() May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._flush(objects) May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application compat.raise_( May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application flush_context.execute() May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 10:39:03.265517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rec.execute(self) May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 10:39:03.267053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application conn.query(q) May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application result.read() May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:39:03.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [parameters: {'id': '9a9cf6af949049f49d350ddc9c07c129', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.270009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:03.271491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9a9cf6af949049f49d350ddc9c07c129. May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 472/946] 10.222.0.36 () {60 vars in 1140 bytes} [Mon May 23 10:39:03 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:03.273142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.311155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.335597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021f9f9-a11b-4b6a-a34a-701844c4ab25 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.365600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.372896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7eba3-0448-4073-9bdc-620e2b6b6aab None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.374401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.375228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.375663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.376099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.376885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] RBAC: Adding query filter params (interface=public, service_id=c69ea0d14a324e049fcde9740976ca58, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:03.377200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.379166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.388657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 475/947] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/endpoints?service_id=c69ea0d14a324e049fcde9740976ca58&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:03.388657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.413815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.414517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.414808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.415092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.415823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:03.416134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.417709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.433164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 473/948] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:03.433991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.495107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7779271-52b3-45eb-9a82-0bf28247efff None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.533611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.534440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.534778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.535087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.541922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.543406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.545544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d252-320a-42b2-9860-effc99816e6b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:03.559066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:03.560301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 476/949] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:03.561194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.564789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.588225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.588890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.589181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.589467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.590197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.591567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.597934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 474/950] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:39:03.598646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.624360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.697544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352161f-d4c9-4f80-b1d8-1c56b5be5939 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.729802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea9295-096f-4ece-93b5-1de967b6d575 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.736831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.738543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.739019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.739478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.747477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.749309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:03.765621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:03.766623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:03.767525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:03.770032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 477/951] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:03.774163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.774962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.775254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.775536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.776328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:03.776643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.778187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.781253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:03.805980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 475/952] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:03.857069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:03.903692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c9d15-9c8e-4ec2-b894-7626e90b6b32 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:03.942233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:03.942921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:03.943207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:03.943494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:03.944311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:03.944621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:03.946159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c25cfe-4ef6-4c4c-b3de-32d6d34013e5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:03.959887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 478/953] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:03.965991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:04.055500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:04.079168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe6533-5732-4b40-bd26-401e18196747 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:04.115985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:04.116883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:04.117168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:04.117445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/086bf8e27f4745c3a01adcfe8d617cf2/roles/75434acb5a4747e3b9e2654f91e27d2e` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:04.119866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=086bf8e27f4745c3a01adcfe8d617cf2, role_id=75434acb5a4747e3b9e2654f91e27d2e)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:04.121372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805d139-d4fa-479c-a2d9-bf34d825e32e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:04.136887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 476/954] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:39:03 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/086bf8e27f4745c3a01adcfe8d617cf2/roles/75434acb5a4747e3b9e2654f91e27d2e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:39:04.958383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c2347-0236-4449-b46b-8f58e1166346 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:04.958907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c2347-0236-4449-b46b-8f58e1166346 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:04.958984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c2347-0236-4449-b46b-8f58e1166346 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:04.960530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 479/955] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:04.965979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3555f99-92d5-4923-8d70-08d6a920d3e2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:04.966372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3555f99-92d5-4923-8d70-08d6a920d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:04.966555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3555f99-92d5-4923-8d70-08d6a920d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:04.984410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e3555f99-92d5-4923-8d70-08d6a920d3e2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:04.985910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e3555f99-92d5-4923-8d70-08d6a920d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:05.054244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 477/956] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:05.076695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c2347-0236-4449-b46b-8f58e1166346 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:05.183194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c2347-0236-4449-b46b-8f58e1166346 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:05.207209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:05.219893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:05.220491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:05.220774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:05.221052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:05.221783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:05.223221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12e9e4-82c5-4a9f-8346-6c348110cf41 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:05.239159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 480/957] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:39:05 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:05.917723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eca14-7dce-46a5-a987-7c1a5c193206 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:05.918110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eca14-7dce-46a5-a987-7c1a5c193206 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:05.918317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8eca14-7dce-46a5-a987-7c1a5c193206 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:05.919965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 478/958] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:05.925798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bde71-b77d-419e-ba6e-40cab1a02c31 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:05.926141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bde71-b77d-419e-ba6e-40cab1a02c31 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:05.926425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bde71-b77d-419e-ba6e-40cab1a02c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:05.944255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f88bde71-b77d-419e-ba6e-40cab1a02c31 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:05.945448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f88bde71-b77d-419e-ba6e-40cab1a02c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:06.030583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 481/959] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:06.053604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8eca14-7dce-46a5-a987-7c1a5c193206 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.123415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.157787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8eca14-7dce-46a5-a987-7c1a5c193206 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.196504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.197251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.197537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.197812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.198524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.199944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.205233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93bfb-5f55-460f-b22b-2411eaeb1e66 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.205537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93bfb-5f55-460f-b22b-2411eaeb1e66 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.205788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93bfb-5f55-460f-b22b-2411eaeb1e66 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.207623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 482/960] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:06.215502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.215941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.216317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:06.216930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:06.218287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:06.219414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:06.221785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 479/961] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:06.230911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.238706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:06.240867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:06.326037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 483/962] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:06.351875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94aba1-c8ff-4c93-9120-04cadb2a40a9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.353295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.390785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.393563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.397904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.398480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.399131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.399857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.400691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:06.401019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.402504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfea78-69d0-40a1-b638-3349c609dd7e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.408050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 480/963] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:06.411706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81e15-0b0c-4433-859b-707b0eb6d082 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.413023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81e15-0b0c-4433-859b-707b0eb6d082 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.413610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81e15-0b0c-4433-859b-707b0eb6d082 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.415538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 481/964] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:06.418394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c81e15-0b0c-4433-859b-707b0eb6d082 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.463764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978f9de-8381-4a58-85e4-1fd8c256ab25 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.499635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.500306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.500588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.500861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.501572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.503054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.504898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING keystone.server.flask.common [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 10:39:06.517452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.sql.core [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 10:39:06.517452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 10:39:06.517452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [parameters: {'user_id': 'bcbdd80c801f4a3f88b24a470c3314fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 10:39:06.517452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91696) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-7028007b-4818-44bc-b83d-7d0cb707d5e0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application conn.query(q) May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result.read() May 23 10:39:06.524911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:39:06.525887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application next(self.gen) May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application yield resource May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application next(self.gen) May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.session.rollback() May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.force_reraise() May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise self.value May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application session.commit() May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 10:39:06.526930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._prepare_impl() May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.session.flush() May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._flush(objects) May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application compat.raise_( May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application flush_context.execute() May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rec.execute(self) May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 10:39:06.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 10:39:06.528813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result = self._query(query) May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application conn.query(q) May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application result.read() May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 10:39:06.529685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bcbdd80c801f4a3f88b24a470c3314fd', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:06.533200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 10:39:06.534945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 484/965] 10.222.0.36 () {60 vars in 1131 bytes} [Mon May 23 10:39:06 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.537782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.539196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c81e15-0b0c-4433-859b-707b0eb6d082 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.548545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:06.550712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:06.572806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.573585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.573940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.574320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.575102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:06.575481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.576856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.581809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 482/966] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:06.581809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.630036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 485/967] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:06.630036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.646336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.670100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.693718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a14bf34-fce0-474d-be22-ef730a91c3a2 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.731494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.732486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.732774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.733299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.742438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.743446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb38708-73db-4d84-a2f4-df3201c8619a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.743871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:06.758139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:06.759278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 483/968] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:06.760236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.783267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.783927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.784211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.785039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.785793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:06.787814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:06.810276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 486/969] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:39:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:06.818135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.878436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123791c3-7735-4522-996c-32fa0b24bfe8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:06.902978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.930174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:06.930990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:06.931277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:06.931554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:06.945721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:06.953385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:06.955202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:06.956368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:06.957626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:06.960875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 484/970] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:06.967699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:06.997841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf70fe-22f7-4622-af73-e67057c432de None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.046325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.046973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.047262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.047532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.048338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:07.049080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.051461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.086397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3ee50-575e-45e4-a9d7-1aa1b79683b8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.092620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 487/971] 10.222.0.36 () {56 vars in 1101 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:07.127165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.127865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.128151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.128470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.129256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:07.129569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.131052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.141265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 485/972] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:07.147216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.276067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ea59-c1b6-48b4-93a5-7f9b5bc24e56 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.313910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.314977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.315425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.315926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.327939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.329440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.333588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:07.351757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:07.352751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:07.353660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:07.353660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:07.353660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:07.353660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:07.353660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 488/973] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:07.358961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.405322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:07.470164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ebcb-e24d-4156-a27f-ecede3821e5a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.502839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.504038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.504324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.504599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.505316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:07.505623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.507026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5f174-24e5-444b-a1ac-82fc11f30c9a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.523855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 486/974] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:07.525069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.530139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e35f-5516-4e8e-9233-9a4caf8d2cb1 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.530370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e35f-5516-4e8e-9233-9a4caf8d2cb1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.530647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0e35f-5516-4e8e-9233-9a4caf8d2cb1 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.532112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 487/975] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:07.536937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.537223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.537499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.553832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:07.555040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:07.625487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 488/976] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:07.630718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1d201-8467-4b47-b514-a77e87ca147e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.647861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.667431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.668248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.668532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.668809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.676762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.678281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.678719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:07.695150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:07.696156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:07.697104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 489/977] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:07.701852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.760328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b965-f75f-49e3-a085-743815d46dd5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.763142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:07.797219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.798043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.798326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.798604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.799254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.801218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.810444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b37874-60ce-4a62-afeb-df329e23e1bc None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:07.813254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:07.814444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:07.815305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 489/978] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:07.820175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.850581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160fa42-a697-423f-8263-b16157023645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.851254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fa42-a697-423f-8263-b16157023645 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.851543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fa42-a697-423f-8263-b16157023645 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.851828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160fa42-a697-423f-8263-b16157023645 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.852668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160fa42-a697-423f-8263-b16157023645 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:07.852984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160fa42-a697-423f-8263-b16157023645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.854469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160fa42-a697-423f-8263-b16157023645 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.876174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 490/979] 10.222.0.36 () {56 vars in 1149 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:07.877243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160fa42-a697-423f-8263-b16157023645 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.921561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:07.931356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5e0b4-bc8f-4f76-9bb2-c8a03ddec1e7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.965900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:07.967002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:07.967295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:07.967575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:07.968434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:07.968745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:07.970138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:07.976007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 490/980] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:07.981253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:07.984422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160fa42-a697-423f-8263-b16157023645 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:07.999801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.024005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.024653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.024938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.025213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.084083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9a0f2-c65c-4e25-b5d7-cc5a7194be30 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.095529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.097685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.119693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.120474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.120800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.121080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.121791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:08.122128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.123485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.128337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 491/981] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:08.134121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.158192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.213053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 491/982] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:07 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:08.214579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.229718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.250781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ebfaa-66b6-4b77-9413-469f7c2b4788 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.289597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.290512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.290811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.291336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.292964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] RBAC: Adding query filter params (interface=public, service_id=b1d20a0022a94f6a8467dc29b6ba904d, region_id=RegionOne) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:08.293313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.294715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.303595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 492/983] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/endpoints?service_id=b1d20a0022a94f6a8467dc29b6ba904d&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:08.308721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.321665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2f86-6143-4a0f-a4a7-36a0a3c15435 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.356829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.357548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.357833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.358151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.368438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.369890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:08.389460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:08.391019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:08.391959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:08.391959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:08.391959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:08.391959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:08.391959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 492/984] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:08.398322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.401285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.421055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3123eb-b1d1-44b5-b4bc-6146e46c3ae8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.456155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.457382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.457662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.457936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.458636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.459993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.467257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 493/985] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:39:08.479040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.519921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83feb6d4-7446-4fad-9415-5f6b28094404 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.557847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.558561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.558851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.559130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.559864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:08.560178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.561624 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.579350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 493/986] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:08.580892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.689114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d9189-b185-4699-9ab6-c9d9294b9b5c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.721992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.722785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.723065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.723336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.729908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.731369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:08.746645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:08.747570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:08.748975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 494/987] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:08.754438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.854477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97d92f-9aab-4381-a74b-6258aaccc59a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:08.880149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:08.892733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:08.893893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:08.894430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:08.894934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:08.896436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:08.896991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:08.899304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da388-95df-4b6e-bac7-f19e224ac60f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:08.917254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 494/988] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:08.918560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:08.932508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:09.024307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba7bb8-4c61-4c75-bb7f-f6a7c74a1079 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.058647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:09.059302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.059586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.059861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.060688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] RBAC: Adding query filter params (role.id=75434acb5a4747e3b9e2654f91e27d2e, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=086bf8e27f4745c3a01adcfe8d617cf2) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:09.061000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:09.062467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537b85-83da-4025-b691-7d648ae05fa6 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:09.074199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 495/989] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:08 2022] GET /identity/v3/role_assignments?user.id=086bf8e27f4745c3a01adcfe8d617cf2&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=75434acb5a4747e3b9e2654f91e27d2e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:09.128558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e70d8-5584-449c-85f2-e368af0c1391 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.129124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e70d8-5584-449c-85f2-e368af0c1391 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.129124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e70d8-5584-449c-85f2-e368af0c1391 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.130610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 495/990] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:09.136684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.137234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.137234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.155225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:09.156483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:09.242366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 496/991] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:09.258532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e70d8-5584-449c-85f2-e368af0c1391 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:09.368899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e70d8-5584-449c-85f2-e368af0c1391 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.403129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:09.403934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.404223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.404509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.405159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:09.406578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:09.418579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:09.419709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:09.420554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 496/992] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:39:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:09.426589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:09.428778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:09.526068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcddb8-7abf-4c51-b87f-8b026928c824 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.559930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:09.560602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.560890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.561171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.561880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:09.562220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:09.563604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:09.568668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 497/993] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:09.574145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:09.628921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:09.681421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7f0a2-0df4-483f-a8e6-d5896d8216b0 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.716246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:09.716909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.717194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.717473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.718201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:09.718511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:09.719891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:09.724945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 497/994] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:09.731001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:09.798410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:09.834276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb40308-b432-4687-bfef-7d2d2364fe6c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.867478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:09.868137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:09.868424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:09.868704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] PATH_INFO: `/v3/endpoints` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:09.869502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] RBAC: Adding query filter params (interface=admin, service_id=c69ea0d14a324e049fcde9740976ca58, region_id=RegionOne) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:09.869818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:09.871212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24b9b3-f82a-4554-a7c6-580c94977cee None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:09.881301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 498/995] 10.222.0.36 () {56 vars in 1249 bytes} [Mon May 23 10:39:09 2022] GET /identity/v3/endpoints?service_id=c69ea0d14a324e049fcde9740976ca58&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:09.882865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:09.984301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403d3815-deb6-41c0-bdbc-c20a36c81425 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:09.997966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:10.019865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:10.020607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:10.020959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:10.021301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:10.022080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:10.023518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4547ad-e59d-46a5-bbbb-ac1fda9e09fe None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:10.030132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 498/996] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:39:09 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:39:10.703260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc12f3-a7f9-4413-a3d6-2958af343100 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:10.703810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc12f3-a7f9-4413-a3d6-2958af343100 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:10.703810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc12f3-a7f9-4413-a3d6-2958af343100 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:10.705348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 499/997] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:10.710935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:10.711365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:10.711512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:10.728377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:10.729704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:10.796395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 499/998] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:10.820159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc12f3-a7f9-4413-a3d6-2958af343100 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:10.923186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc12f3-a7f9-4413-a3d6-2958af343100 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:10.953427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:10.960672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:10.961469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:10.961751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:10.962050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] PATH_INFO: `/v3/services/compute` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:10.962704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:10.964063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:10.976314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:10.977322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 10:39:10.978582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:10.982041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 500/999] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:10.984029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.082687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de76888-0b60-42b2-9e85-28063c7a3a88 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.092568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:11.121590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.122310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.122596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.122874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.123602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] RBAC: Adding query filter params (name=compute) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:11.123993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.125443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdb8b6-e42a-43de-ab9e-460b92c3b73c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.136173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 500/1000] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:11.137573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.248699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecd4fb-351c-4987-b64c-b360f7bd56db None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.284138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.284830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.285114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.285389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.286130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:11.286437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.287862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2be3f7-3f2a-4d3f-a340-fb254f1e50ce None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.292169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc150-0bfa-432c-9e29-5b43a33286db None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.292462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc150-0bfa-432c-9e29-5b43a33286db None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.292743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc150-0bfa-432c-9e29-5b43a33286db None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.293268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 501/1001] 10.222.0.36 () {56 vars in 1112 bytes} [Mon May 23 10:39:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:11.294159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 501/1002] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:11.299225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.299423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fc150-0bfa-432c-9e29-5b43a33286db None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.299524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.299839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.320889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:11.322632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:11.387750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:11.403929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 502/1003] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:11.404467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fc150-0bfa-432c-9e29-5b43a33286db None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.425638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.440143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.441345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.441628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.441902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.442659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.444044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.464797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 502/1004] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:39:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:39:11.535437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9cee-f387-4c18-a966-565082e82da6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.579016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.579820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.580101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.580371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] PATH_INFO: `/v3/services/image` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.581022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.582475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:11.594696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:11.595805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:11.602422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 503/1005] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 23 10:39:11 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:11.603255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.634644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:11.713835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa14a7c-bfae-472e-9c1d-28b27db5c917 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.748513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.749186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.749486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.749765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.750504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] RBAC: Adding query filter params (name=image) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:11.750832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.752199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07750c-1ae3-4a6a-9ad4-9cbf21ed7267 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.758143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 503/1006] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:11.763728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:11.815391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:11.869767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f7606-4f27-4ef9-b42e-c4aa2cbe364b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:11.905003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:11.905668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:11.905967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:11.906257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:11.906963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] RBAC: Adding query filter params (type=image) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:11.907270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:11.908695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223392-99ff-4f65-a30a-92fbf1b96109 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:11.914501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 504/1007] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:12.252376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3da368-6f3a-4292-bf7d-bcd3bcd5da92 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.252659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3da368-6f3a-4292-bf7d-bcd3bcd5da92 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.252941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3da368-6f3a-4292-bf7d-bcd3bcd5da92 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.255064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 504/1008] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:12.260150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607eba8-0464-451d-a913-251e1562fedf None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.260437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607eba8-0464-451d-a913-251e1562fedf None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.260712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607eba8-0464-451d-a913-251e1562fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.278219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e607eba8-0464-451d-a913-251e1562fedf None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:12.279407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e607eba8-0464-451d-a913-251e1562fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:12.348321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 505/1009] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:12 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:12.370714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3da368-6f3a-4292-bf7d-bcd3bcd5da92 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:12.461640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:12.472093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3da368-6f3a-4292-bf7d-bcd3bcd5da92 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:12.506159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:12.506895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.507179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.507455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.508098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:12.509467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:12.521689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:12.522686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 10:39:12.523527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:12.526266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 505/1010] 10.222.0.36 () {56 vars in 1126 bytes} [Mon May 23 10:39:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:12.528809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607eba8-0464-451d-a913-251e1562fedf None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:12.572793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:12.630602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607eba8-0464-451d-a913-251e1562fedf None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:12.664396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:12.665067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.665349 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.665622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.666357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:12.666662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:12.668058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:12.678686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 506/1011] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:12.680023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:12.782522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608949bb-7240-46c3-823e-58ac4f297d45 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:12.816451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:12.817104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.817387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.817661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.818380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:12.818692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:12.820056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6c195-a82c-4920-8923-cb1645e11f3d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:12.824959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 506/1012] 10.222.0.36 () {56 vars in 1122 bytes} [Mon May 23 10:39:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:12.830524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:12.935569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e762b-b48b-4fb7-8f2e-17b734a843cd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:12.943580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:12.972973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:12.973627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:12.973909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:12.974213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] PATH_INFO: `/v3/endpoints` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:12.974958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:12.976589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3aee78-cc41-4836-a06d-b5c048fe7aae None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:12.997979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 507/1013] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:39:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:13.768095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb15d08-f090-4513-b0c5-7be949c81b41 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:13.768491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb15d08-f090-4513-b0c5-7be949c81b41 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:13.769254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb15d08-f090-4513-b0c5-7be949c81b41 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:13.770764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 507/1014] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:13.775843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:13.776134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:13.776412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:13.792987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:13.794198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:13.868635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 508/1015] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:13 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:13.892056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb15d08-f090-4513-b0c5-7be949c81b41 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:13.914583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:13.999537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb15d08-f090-4513-b0c5-7be949c81b41 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.032946 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.033678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.033980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.034297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.044734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-2612d6b9-99b2-4104-aa91-66ee0f1ed603 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:14.053323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:14.054476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:14.055335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:14.056465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 508/1016] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:39:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:14.060382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.131258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cb1e-eb87-48a7-86f7-7d1e7f341942 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.131780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cb1e-eb87-48a7-86f7-7d1e7f341942 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.132065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1cb1e-eb87-48a7-86f7-7d1e7f341942 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.133591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 509/1017] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:14.138377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.138670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.138951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.156467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:14.156666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:14.157676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:14.166685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf74e-1678-4485-914f-4e1e2d1dd8b5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.204908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381910be-3581-4aa8-a59c-55168834952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.205877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381910be-3581-4aa8-a59c-55168834952e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.206498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381910be-3581-4aa8-a59c-55168834952e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.206926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381910be-3581-4aa8-a59c-55168834952e None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.207936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381910be-3581-4aa8-a59c-55168834952e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:14.208397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381910be-3581-4aa8-a59c-55168834952e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.210070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381910be-3581-4aa8-a59c-55168834952e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.224680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 509/1018] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:14.225527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381910be-3581-4aa8-a59c-55168834952e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.232606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 510/1019] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:14.255815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.267730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:14.335331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381910be-3581-4aa8-a59c-55168834952e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.360906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96803241-13ad-4253-a117-86f780c71bb8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.371987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.373109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.373573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.374096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.385453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.387290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.395760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.396417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.396704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.396983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.397696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.399077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:14.407082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:14.408111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:14.409026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:14.409026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:14.409026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:14.409026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:14.414027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 510/1020] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:14.414978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.418156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 511/1021] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:39:14 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:39:14.426919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:14.529598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d211d3-79c5-4340-aa02-f6b44b5a28a0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.572573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.573261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.573544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.573817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.574580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:14.574890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.576277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.594075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 511/1022] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:14.595901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.669835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:14.702790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d4735-21cf-407a-8b6f-f1459caec555 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.737194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.738042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.738331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.738609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.746337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.747788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:14.761559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:14.762677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:14.769163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 512/1023] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:14.771368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:14.833234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:14.876957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da7fa-8ec9-4fea-a83e-969c0817ce5a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:14.912711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:14.913383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:14.913665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:14.913943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:14.914803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:14.915112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:14.916536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af75ae29-8808-4e18-85e4-d11c28b50c65 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:14.935138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 512/1024] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:14.941444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.058793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456c83b-18a2-4648-9f3a-0542f09f1104 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.087640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:15.094655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.095306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.095588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.095861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.163278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.164636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.197282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06047e-0b73-49eb-89da-36154f0c945c None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.197579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06047e-0b73-49eb-89da-36154f0c945c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.197861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06047e-0b73-49eb-89da-36154f0c945c None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.199427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 513/1025] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:15.204212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.204501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.204781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.221616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:15.222864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:15.280490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 513/1026] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 10:39:15.281851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.298423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 514/1027] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:15.314358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:15.321928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.374812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:15.390658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e024d-c715-40eb-be5c-f6d9e93557ff None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.426729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.427363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b0d89-be21-4ff6-b963-79de2e7da57e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.428020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.428498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.428992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.440151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.444132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:15.463340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:15.464473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:15.465571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:15.465571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:15.465571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:15.465571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:15.471902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 514/1028] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:15.473429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.476750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.477559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.477841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.478267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.479454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.480901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.488033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 515/1029] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:15.488842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.572938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:15.598188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e79d-b1f5-48d0-a6ec-947bfa72facc None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.635746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f457b4-016c-4c52-b786-a8bb9c3236e8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.636227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.636962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.637368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.637710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.638644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:15.639059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.640960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.658715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 515/1030] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:15.660234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.661991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:15.680125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.681056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.681551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.681829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.682590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.684128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.721021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 516/1031] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:39:15 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:15.780702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1933b-aeeb-4065-8210-9e4ba95f018e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:15.826047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:15.826863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:15.827149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:15.827420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:15.835199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:15.836730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:15.852836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:15.853820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:15.854772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:15.859110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 516/1032] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:15.867101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:15.994956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436be5b-a3b5-4d04-b3e0-0395b1030bbf None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:16.028946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:16.029612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:16.029897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:16.030200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:16.030987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:16.031296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:16.032727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef7a7-10b0-44e4-8056-b6ae1b028bb9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:16.045871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 517/1033] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:16.052691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:16.055700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:16.155266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7ae8c-f05a-46fa-b85f-22b0a64fa1b4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:16.189257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:16.189913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:16.190224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:16.190548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:16.191382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] RBAC: Adding query filter params (role.id=75434acb5a4747e3b9e2654f91e27d2e, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:16.191696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:16.193138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b7f76-20a8-4fdb-ac04-a30d2c787cfe None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:16.205991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 517/1034] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:16 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=75434acb5a4747e3b9e2654f91e27d2e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:16.638237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554f847-a997-458c-97e4-b938cfdd6893 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:16.638661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554f847-a997-458c-97e4-b938cfdd6893 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:16.638857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554f847-a997-458c-97e4-b938cfdd6893 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:16.640605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 518/1035] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:16.646173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:16.646465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:16.646748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:16.663372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:16.664566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:16.724512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:16.733379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 518/1036] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:16.755934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554f847-a997-458c-97e4-b938cfdd6893 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:16.786781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:16.859991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554f847-a997-458c-97e4-b938cfdd6893 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:16.898233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:16.898947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:16.899233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:16.899507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] PATH_INFO: `/v3/services/image` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:16.900156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:16.901537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:16.913376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:16.914406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:16.915251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:16.916611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 519/1037] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 23 10:39:16 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:16.920646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:17.026545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91063ed9-2b69-4767-a480-1a2a5bbf4931 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:17.059698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:17.060373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.060630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.060905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:17.061598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] RBAC: Adding query filter params (name=image) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:17.061902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:17.063302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:17.073270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 519/1038] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:17.074453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:17.168737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:17.176852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338b466-2993-4365-91c7-edf90299e330 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:17.210484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:17.211156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.211442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.211718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:17.212424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] RBAC: Adding query filter params (type=image) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:17.212732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:17.214122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:17.223803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 520/1039] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:17.225224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:17.286880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:17.335394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681e41a-693f-48e4-9a5a-db984d568b49 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:17.370448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:17.371102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.371383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.371656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:17.372437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] RBAC: Adding query filter params (interface=public, service_id=50b92d039569465bbd74080ae2dc452b, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:17.372746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:17.374138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bd55b-eb80-42d0-aff9-8b94ce24bc17 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:17.382320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 520/1040] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:39:17 2022] GET /identity/v3/endpoints?service_id=50b92d039569465bbd74080ae2dc452b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:17.385316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:17.489570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f33a5-6a23-4bef-a588-2761ea927160 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:17.524804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:17.525469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.525752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.526055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:17.526737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:17.528218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c852e-0ed2-4dbb-9a53-941d83613625 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:17.535257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 521/1041] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:39:17 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:39:17.986843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705fd8-249b-43b8-beba-eb5d4314f12a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.987280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705fd8-249b-43b8-beba-eb5d4314f12a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.987461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d705fd8-249b-43b8-beba-eb5d4314f12a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:17.988938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 521/1042] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:17.994373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:17.994666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:17.994944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.012352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:18.013539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:18.031872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:18.084165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 522/1043] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:18.107980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705fd8-249b-43b8-beba-eb5d4314f12a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:18.212983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d705fd8-249b-43b8-beba-eb5d4314f12a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:18.246741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:18.258801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:18.259604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.259887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.260163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.261335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:18.262824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f58444-1275-44b3-93d3-941f97c9856d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:18.269652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 522/1044] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:18.270937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:18.369728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0eee2-c4f5-479f-ab98-a768f6b98acd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:18.387951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c221-85ad-4d9c-bf33-41f6c2691a55 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.388244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c221-85ad-4d9c-bf33-41f6c2691a55 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.388524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5c221-85ad-4d9c-bf33-41f6c2691a55 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.389903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 523/1045] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:18.395190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.395484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.395766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.407716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:18.408646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.409134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.409584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.410494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:18.412090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:18.412466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:18.413665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:18.435822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 523/1046] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:39:18 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:39:18.489761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 524/1047] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:18.511609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:18.523973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:18.623009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad92d11-4b6b-40f1-839f-115cca88ec1a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:18.661172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:18.662416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.662702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.662981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.673382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:18.682219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:18.683364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:18.684456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 524/1048] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:39:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:18.690020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:18.796036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198bb30-69ab-49a5-8306-45e90fde82e6 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:18.819424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:18.831550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:18.832213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:18.832494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:18.832766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:18.833505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:18.833808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:18.835204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:18.843612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 525/1049] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:18.850202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:18.927936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:18.962557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9867c-db36-4313-b75e-a541fc85f46f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.003792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.004583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.004864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.005138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.018162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.019634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:19.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:19.037897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:19.043578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:19.043578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:19.043578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:19.043578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:19.043578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 525/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:19.043872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.143177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8c14-cf49-4ddb-b3c5-88f4eac497c2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.176973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.177629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.177910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.178214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.178925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:19.179230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.180603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.197508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 526/1051] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:19.199049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.307068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649176c-0ce0-4471-ac8f-2dc10af80c85 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.327826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:19.343397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.344195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.344481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.344760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.353201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.354686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:19.368535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:19.369458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:19.370451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:19.371566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 526/1052] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:19.375427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.423008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:19.478700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2d45a-480a-407c-85cf-7d61257f2a9f None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.511697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.512354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.512637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.512913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.513735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:19.514069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.515770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.534160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 527/1053] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:19.539565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.644948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f535687-a3be-481e-b31b-3949a5db1b4a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.678157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.678871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.679152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.679424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.689976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.691990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-5895ca2d-963c-4e5f-9e64-21ca1b388376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:19.709463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:19.710642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:19.711546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:19.711546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:19.711546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:19.711546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:19.714031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 527/1054] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:19.716379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.739416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d144-0601-4770-802a-19270648b3fe None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.739712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d144-0601-4770-802a-19270648b3fe None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.739996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85d144-0601-4770-802a-19270648b3fe None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.741365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 528/1055] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:19.746325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.746614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.746915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.764715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:19.766204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:19.793521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:19.825553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4f52-dde1-4ae9-a104-89028c6dcbe2 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.827217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:19.839654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 529/1056] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:19.861732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:19.862043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.862963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:19.863242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:19.863512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:19.864222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:19.864654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:19.866158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:19.883690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 528/1057] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:19.884411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:19.966550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11ad21-8f08-4f53-9a9e-fb43ed1953c9 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:19.989862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b90172-8658-494a-b23d-1b2afd771110 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.004053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.004841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.005122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.005395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] PATH_INFO: `/v3/services/image` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.006067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.007436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:20.019354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:20.025570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:20.026662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 530/1058] 10.222.0.36 () {56 vars in 1105 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:20.027548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.033786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.034647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.034931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.035201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.042117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.043563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.056884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:20.058459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:20.059601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:20.061304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 529/1059] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:20.066046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.090974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:20.137182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729488ce-9565-4c18-9d80-e90102208a65 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.171664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6525-0a60-42bd-8b4b-278985a7150c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.176649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.177320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.177600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.177874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.178612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] RBAC: Adding query filter params (name=image) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:20.178920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.180698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.192080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 531/1060] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:20.193079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.207269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.208150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.208485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.208776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.209605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:20.209929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.211416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.226049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 530/1061] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:20.229463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.303969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:20.316493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fed65b-5862-463d-b05b-d7121fae6a03 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.323124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:20.342517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c8136-ef2c-4313-bcde-9cf844c4f8d3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.355916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.356815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.357465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.357747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.358751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] RBAC: Adding query filter params (type=image) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:20.359059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.360883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.366957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 532/1062] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:20.372650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.382122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.383041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.383328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.383606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/75434acb5a4747e3b9e2654f91e27d2e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.385850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=8344a4e41ea548568e99784e3a6ef493, role_id=75434acb5a4747e3b9e2654f91e27d2e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.387326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a9a815-3405-4f25-8ee3-aa168c53bca5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.401996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 531/1063] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:39:20 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/8344a4e41ea548568e99784e3a6ef493/roles/75434acb5a4747e3b9e2654f91e27d2e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:39:20.486934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd1ce3-3187-42ef-b649-638719e8852b None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:20.523266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:20.523942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.524228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.524505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.525264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:20.526727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f4b73c-3270-4efc-8114-ce0bc34edf02 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:20.558006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 533/1064] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:39:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:39:20.739880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd793a0-0527-4941-8429-f95257e87374 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.740094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd793a0-0527-4941-8429-f95257e87374 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.740375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd793a0-0527-4941-8429-f95257e87374 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.741768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 532/1065] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:20.747134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:20.747434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:20.747718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:20.765984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:20.767175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:20.771676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:20.847479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 534/1066] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:20.870141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd793a0-0527-4941-8429-f95257e87374 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:20.975242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd793a0-0527-4941-8429-f95257e87374 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.011631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.012344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.012628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.012905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.024010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:21.031618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:21.032551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:21.033405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:21.034781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 533/1067] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:21.038998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:21.139097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b785f82-2009-4f28-8513-e4356c3f2b9a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.173617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.174320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.174640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.174926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.175674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:21.175983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:21.177361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:21.179639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:21.190045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 535/1068] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:21.193648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:21.196632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:21.303881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169c1db-f735-486c-b719-7a14e62ee5da None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.338644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.339772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.340239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.340696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] PATH_INFO: `/v3/users/cd28c29c284148f6a613b0f10663bf20` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.354235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] RBAC: Authorizing `identity:get_user(user_id=cd28c29c284148f6a613b0f10663bf20)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:21.355914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:21.362654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 534/1069] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:21 2022] GET /identity/v3/users/cd28c29c284148f6a613b0f10663bf20 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:21.364510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:21.465776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf7241-d844-42cb-b655-87287eaaa4a0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.499122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.500018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.500485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.500941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.570061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:21.573105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:21.574851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:21.673356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 536/1070] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:21.679669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:21.740493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:21.782876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb418f5-90f2-4b24-8555-9e85f0c022fa None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.818216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.819408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.819898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.820431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.821819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:21.823548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a89eba-353e-46b0-ae70-fabe093617e0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:21.825746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 535/1071] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:21 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:21.831556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:21.929397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:21.933809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27695cba-a093-449f-981d-0d51aea799b0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:21.967399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361664c1-5282-497a-a296-141426c4e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:21.968050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361664c1-5282-497a-a296-141426c4e315 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:21.968336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361664c1-5282-497a-a296-141426c4e315 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:21.968613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361664c1-5282-497a-a296-141426c4e315 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:21.969438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361664c1-5282-497a-a296-141426c4e315 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=71e7caaaa0334938a02dd6939319e82c, user.id=cd28c29c284148f6a613b0f10663bf20) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:21.969753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361664c1-5282-497a-a296-141426c4e315 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:21.971212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361664c1-5282-497a-a296-141426c4e315 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:21.977754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 537/1072] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:21 2022] GET /identity/v3/role_assignments?user.id=cd28c29c284148f6a613b0f10663bf20&scope.project.id=71e7caaaa0334938a02dd6939319e82c&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:22.615713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75e56-a226-422a-8644-a9cfff7a414c None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:22.616272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75e56-a226-422a-8644-a9cfff7a414c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:22.616272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee75e56-a226-422a-8644-a9cfff7a414c None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:22.617672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 536/1073] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:22.623597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156f9a-f70f-4a1a-a54d-5c1f094a6963 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:22.624060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156f9a-f70f-4a1a-a54d-5c1f094a6963 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:22.624351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52156f9a-f70f-4a1a-a54d-5c1f094a6963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:22.641926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-52156f9a-f70f-4a1a-a54d-5c1f094a6963 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:22.643148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-52156f9a-f70f-4a1a-a54d-5c1f094a6963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:22.710638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 538/1074] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:22.733656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee75e56-a226-422a-8644-a9cfff7a414c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:22.783955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef185-fac1-40ce-bd60-3fcc9cac882b None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:22.784298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef185-fac1-40ce-bd60-3fcc9cac882b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:22.784531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ef185-fac1-40ce-bd60-3fcc9cac882b None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:22.785906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 539/1075] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:22.790689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:22.790975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:22.791256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:22.808124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:22.809306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:22.845799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee75e56-a226-422a-8644-a9cfff7a414c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:22.874208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:22.882743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 540/1076] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:22.883493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:22.884307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:22.884590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:22.884866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:22.895593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:22.903766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:22.904719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 10:39:22.905598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:22.906426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:22.906501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 537/1077] 10.222.0.36 () {56 vars in 1120 bytes} [Mon May 23 10:39:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:22.910642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:22.932987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.009703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdd656-1bf8-4366-a1d3-7c1e272f9bc9 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.022197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae87bfc-deff-419c-92f8-bcf765bea2f9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.047259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.047900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.048183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.048458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.060278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.061398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.061690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.062341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.063127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:23.063435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.064851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.078785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 538/1078] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:23.079801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.125058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.126502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.137017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.152243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.193898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895ae79-813c-467a-b319-dc433e712c0c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.229700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 541/1079] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 10:39:23.235261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.235518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.236460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.236749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.237074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.249817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.251422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:23.273848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:23.274863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:23.275762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:23.275762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:23.275762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:23.275762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:23.275762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 539/1080] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:23.280675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.341123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39af1f-155f-4e8a-9cc2-b05ca745b2de None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.365833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.379691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.380411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.380695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.380969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.384859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a746e-ff8d-46f1-b7e7-0080efbf9ef8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.386567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.392092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.393585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:23.412467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:23.413393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:23.414959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:23.414959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:23.414959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:23.414959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:23.414959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 542/1081] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:23.419369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.424520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.425773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.426160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.426436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.427146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:23.427449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.428808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.444382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 540/1082] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:23.445380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.529224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907c26-2427-4ab6-9d8c-d740ee9408c7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.556500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c41060-6f78-487c-b5c9-331ed6e332eb None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.565703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.566401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.566685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.566970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.567696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:23.568004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.569405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.582255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 543/1083] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:23.607456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.608269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.608556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.608831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.617933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.619459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:39:23.635054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:23.636083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:23.642644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 541/1084] 10.222.0.36 () {56 vars in 1093 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:23.645533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.683124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.755175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a27d2b-508e-4ec3-83ed-a3e0c0172635 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.794881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.795565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.795854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.796133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:23.796991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:23.797307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:23.798834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:23.819530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 544/1085] 10.222.0.36 () {56 vars in 1133 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:23.825088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:23.853399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:23.942536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520780-2505-468d-8390-f9aefa30d9a5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:23.980707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:23.981508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:23.981868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:23.982237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.052395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.053865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53325340-c1af-42b7-834d-e745bc1b451d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.077373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.152495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 542/1086] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:24.158530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.170135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca577-8499-427f-88f6-f8394756732d None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.170428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca577-8499-427f-88f6-f8394756732d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.170706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca577-8499-427f-88f6-f8394756732d None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.172074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 543/1087] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:39:24.176975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.177258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.177539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.195920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:24.197059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:24.249534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.259544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1f10f-1555-4a17-8234-94f438e16fd4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.266246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 544/1088] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:39:24.290039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.295324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.296129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.296415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.296691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.307605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.309208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:24.327519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:24.328691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:24.329692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:39:24.329692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:24.329692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:39:24.329692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:24.329692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 545/1089] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:24.335187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.343061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.390506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec75d9-4ccb-4117-adde-d63e2f354068 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.425646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.426463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.426741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.427010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.434701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f3588-6746-4d55-bbb7-21fdd82f808c None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.437601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:24.445096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:24.446136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:24.446992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 545/1090] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:24.452282 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.473421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.474125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.474412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.474688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.475400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:24.475702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.477072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.488132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 546/1091] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.493525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.497616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.551581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8d8a-5db9-4603-84c5-e91fa5448e80 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.564012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.586327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.587022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.587305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.587813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.588316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:24.588625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.590055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.590344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41cf4f-aa8f-471b-bc40-8c6ef39053ff None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.598549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 546/1092] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.604210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.623997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.624777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.625056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.625332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.631898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.633334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:39:24.647876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:24.649408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:24.650623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 547/1093] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:39:24.655000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.702567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c6c0-0e11-4ee7-891c-749f009452bf None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.716701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.736264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.737128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.737408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.737678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] PATH_INFO: `/v3/users/cd28c29c284148f6a613b0f10663bf20` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.738896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cd28c29c284148f6a613b0f10663bf20)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.740320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.743009 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 547/1094] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/users/cd28c29c284148f6a613b0f10663bf20 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.747457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.753787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fba9f-6171-4659-b261-7ef32c902eaf None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.787918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.788577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.788860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.789137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.789749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.789914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:24.790253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.792289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.803711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 548/1095] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.810270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.846024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43838b8-7d49-44e6-88c1-791d825021b1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.878918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.879774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.880055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.880328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.881494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.882961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.885744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 548/1096] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.890814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:24.909241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e331c-7643-416f-b325-c7f12ed326b0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:24.936146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:24.948250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:24.948903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:24.949185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:24.949460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:24.950315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] RBAC: Adding query filter params (role.id=75434acb5a4747e3b9e2654f91e27d2e, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=8344a4e41ea548568e99784e3a6ef493) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:24.950627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:24.952059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eff99e-ab40-4b98-a21c-860d0ff38ede None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:24.958482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 549/1097] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:24 2022] GET /identity/v3/role_assignments?user.id=8344a4e41ea548568e99784e3a6ef493&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=75434acb5a4747e3b9e2654f91e27d2e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:39:24.993887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56e789-7d54-4816-9de8-dff020458057 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:25.005878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:25.032088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:25.032983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:25.033263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:25.033538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c/users/cd28c29c284148f6a613b0f10663bf20/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:25.035666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e7caaaa0334938a02dd6939319e82c, user_id=cd28c29c284148f6a613b0f10663bf20, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:25.037502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35129f-ada4-4e2f-a50d-9caa0ae6ccd4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:25.052003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 549/1098] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:39:24 2022] PUT /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c/users/cd28c29c284148f6a613b0f10663bf20/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:39:27.208566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258858a-d32f-44f9-b502-daf5c2d14b49 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.209120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258858a-d32f-44f9-b502-daf5c2d14b49 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.209120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258858a-d32f-44f9-b502-daf5c2d14b49 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.210571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 550/1099] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:27.215954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.216376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.216510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.233408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:27.234623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:27.298735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 550/1100] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:27.322880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258858a-d32f-44f9-b502-daf5c2d14b49 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:27.420372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258858a-d32f-44f9-b502-daf5c2d14b49 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:27.453085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:27.453864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.454181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.454461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.465358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:27.473004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:27.474145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:27.475483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 551/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:27.479943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:27.571620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:27.574586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23ebe-1183-4f0b-871b-871afb5e9af7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:27.607402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:27.608044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.608323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.608596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.609326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:27.609630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:27.611019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:27.619263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 551/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:27.624865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:27.672682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:27.721088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec65d6-cb1b-4c79-b735-292b9fc3a2a5 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:27.754109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:27.754974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.755255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.755527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] PATH_INFO: `/v3/users/cd28c29c284148f6a613b0f10663bf20` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.756699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] RBAC: Authorizing `identity:get_user(user_id=cd28c29c284148f6a613b0f10663bf20)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:27.758165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:27.759780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 552/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:27 2022] GET /identity/v3/users/cd28c29c284148f6a613b0f10663bf20 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:27.765626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:27.860799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79760de-3db4-4778-bde6-a8e4ef6a7aea None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:27.893232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:27.893856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:27.894162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:27.894436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:27.923006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:27.959697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:27.961123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:28.057936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 552/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:28.063646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:28.158103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8797820-f484-44aa-82cb-8ce43b146f70 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:28.174163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:28.191399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:28.192243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:28.192522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:28.192795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:28.193985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:28.195414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb2c3b-264e-4974-9738-646c0908f7b1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:28.197077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 553/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:28 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:28.203109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:28.261893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:28.299585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfab60d-a371-4422-9760-063a72b6c239 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:28.331782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:28.332361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:28.332642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:28.332915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:28.333728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=71e7caaaa0334938a02dd6939319e82c, user.id=cd28c29c284148f6a613b0f10663bf20) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:28.334067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:28.335501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17d7ec-90d0-46f6-bdd5-a4a5b620ef18 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:28.341098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 553/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:28 2022] GET /identity/v3/role_assignments?user.id=cd28c29c284148f6a613b0f10663bf20&scope.project.id=71e7caaaa0334938a02dd6939319e82c&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:30.527961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132114-b843-4fbf-8d7b-72c4953bf1d5 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:30.528477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132114-b843-4fbf-8d7b-72c4953bf1d5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:30.528477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77132114-b843-4fbf-8d7b-72c4953bf1d5 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:30.529793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 554/1107] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:30.535037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:30.535543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:30.535667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:30.552444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:30.553603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:30.617592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 554/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:30.641651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77132114-b843-4fbf-8d7b-72c4953bf1d5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:30.738969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77132114-b843-4fbf-8d7b-72c4953bf1d5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:30.771026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:30.771795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:30.772078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:30.772354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:30.773508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:30.774966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea13251-d230-4318-936d-70e6496d6cef None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:30.776552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 555/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:30.782013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:30.834512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:30.876234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98154b-61b3-4784-b9ec-a7dd4f541457 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:30.908234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:30.909415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:30.909692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:30.909982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:30.910678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:30.912094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd8668-acad-4360-afc0-5dc9e7c80ab2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:30.932079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 555/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:39:30 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:33.120184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf928-17dc-4943-81d2-ca827f227825 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.120751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf928-17dc-4943-81d2-ca827f227825 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.120751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cf928-17dc-4943-81d2-ca827f227825 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.122135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 556/1111] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:33.129156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.129597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.129722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.147068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:33.148255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:33.211885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 556/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:33.234599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cf928-17dc-4943-81d2-ca827f227825 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:33.287887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:33.334807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cf928-17dc-4943-81d2-ca827f227825 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:33.367263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:33.368104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.368390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.368669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] PATH_INFO: `/v3/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.370159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] RBAC: Authorizing `identity:get_role(role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:33.371573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:33.372888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 557/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:33 2022] GET /identity/v3/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:33.379132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:33.420013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:33.475211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b3622-b91f-4e0f-b11e-9d2fc0e1d700 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:33.508038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:33.508897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.509179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.509456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] PATH_INFO: `/v3/users/3ca6592b46d6427098e2c764c9f6aac6` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.522709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] RBAC: Authorizing `identity:get_user(user_id=3ca6592b46d6427098e2c764c9f6aac6)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:33.524167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:33.525798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 557/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:33 2022] GET /identity/v3/users/3ca6592b46d6427098e2c764c9f6aac6 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:33.531441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:33.626078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dfa36-6c1b-43ed-99ce-65a0d2110875 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:33.645516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:33.659553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:33.660183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.660462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.660733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.724120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:33.725563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:33.818912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 558/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:33.824517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:33.918263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:33.921264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29570cf5-baec-4775-ab9a-2b9f7ab3f0e4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:33.954861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:33.956318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:33.956604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:33.956886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:33.958109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:33.959555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed16f6-3baf-48b4-8737-cb1077a80b94 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:33.961278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 558/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:33 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:33.967193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:33.987357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:34.067525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bc4d9-8026-4968-8c8d-eec4c868adfd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:34.099044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:34.099678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:34.099958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:34.100229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:34.101043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=71e7caaaa0334938a02dd6939319e82c, user.id=3ca6592b46d6427098e2c764c9f6aac6) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:34.101351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:34.102794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09689429-b3b7-4cc0-8755-b7b106c64a9e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:34.108008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 559/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:33 2022] GET /identity/v3/role_assignments?user.id=3ca6592b46d6427098e2c764c9f6aac6&scope.project.id=71e7caaaa0334938a02dd6939319e82c&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:36.237765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d8de-0633-4b84-960d-5032d5c94d9d None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.238382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d8de-0633-4b84-960d-5032d5c94d9d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.238382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d8de-0633-4b84-960d-5032d5c94d9d None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.239718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 559/1118] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:36.245048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.245430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.245622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.262502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:36.263685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:36.328759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 560/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:36.351651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533d8de-0633-4b84-960d-5032d5c94d9d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:36.449003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533d8de-0633-4b84-960d-5032d5c94d9d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:36.480832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:36.481667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.481972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.482249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] PATH_INFO: `/v3/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.483710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] RBAC: Authorizing `identity:get_role(role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:36.485108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:36.487028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 560/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:36 2022] GET /identity/v3/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:36.492596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:36.507689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:36.588848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c0962-02d6-4b91-8a09-dceb29f5c4c6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:36.621744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:36.622626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.622912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.623188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] PATH_INFO: `/v3/users/3ca6592b46d6427098e2c764c9f6aac6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.624376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] RBAC: Authorizing `identity:get_user(user_id=3ca6592b46d6427098e2c764c9f6aac6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:36.625815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:36.628059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 561/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:36 2022] GET /identity/v3/users/3ca6592b46d6427098e2c764c9f6aac6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:36.633731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:36.667861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:36.731003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c37c6b-d8cf-49c3-bd92-8b8b7251b6ca None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:36.763862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:36.764641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.764923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.765449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.766770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:36.768279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae2276-4ce5-42c8-bdc3-441acea33562 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:36.769967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 561/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:36 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:36.775609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:36.853566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:36.871485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb94f5-0550-4e23-af8a-fd70a781f86e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:36.903669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:36.904533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:36.904813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:36.905085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c/users/3ca6592b46d6427098e2c764c9f6aac6/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:36.907263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] RBAC: Authorizing `identity:create_grant(project_id=71e7caaaa0334938a02dd6939319e82c, user_id=3ca6592b46d6427098e2c764c9f6aac6, role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:36.909174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b412c-bd32-4855-96b7-713db000c010 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:36.921342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 562/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:39:36 2022] PUT /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c/users/3ca6592b46d6427098e2c764c9f6aac6/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:39:39.061378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533d44-d9ef-4dcb-a47a-6e82fc5b8f31 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.061938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533d44-d9ef-4dcb-a47a-6e82fc5b8f31 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.061938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533d44-d9ef-4dcb-a47a-6e82fc5b8f31 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.063323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 562/1124] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:39.068553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.068955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.069108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.085286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:39.086482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:39.162014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 563/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:39.185319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4533d44-d9ef-4dcb-a47a-6e82fc5b8f31 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:39.283284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4533d44-d9ef-4dcb-a47a-6e82fc5b8f31 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:39.285537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:39.316643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:39.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.317775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.318073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] PATH_INFO: `/v3/roles/a08f0a1b75f34a20a379690997b9cf81` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.319530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] RBAC: Authorizing `identity:get_role(role_id=a08f0a1b75f34a20a379690997b9cf81)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:39.320899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:39.322287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 563/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:39 2022] GET /identity/v3/roles/a08f0a1b75f34a20a379690997b9cf81 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:39.328119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:39.395644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:39.423512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b720-6a59-4ed9-83c3-261884ef929f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:39.456106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:39.456959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.457240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.457511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] PATH_INFO: `/v3/users/3ca6592b46d6427098e2c764c9f6aac6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.458699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] RBAC: Authorizing `identity:get_user(user_id=3ca6592b46d6427098e2c764c9f6aac6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:39.460128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:39.461717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 564/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:39 2022] GET /identity/v3/users/3ca6592b46d6427098e2c764c9f6aac6 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:39.467372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:39.564331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdbc7-da43-4bde-aad1-6f714b232db4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:39.596969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:39.597540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.597828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.598141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.637541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:39.663630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:39.665076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:39.759142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 564/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:39.764876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:39.860097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995c64a-b464-4790-935b-882c6be0b940 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:39.892837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:39.893671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:39.893971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:39.894251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] PATH_INFO: `/v3/projects/71e7caaaa0334938a02dd6939319e82c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:39.895415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] RBAC: Authorizing `identity:get_project(project_id=71e7caaaa0334938a02dd6939319e82c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:39.896831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff616a-35eb-4599-b880-4b8824e78e3a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:39.898513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 565/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:39 2022] GET /identity/v3/projects/71e7caaaa0334938a02dd6939319e82c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:39.904319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:39.972209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:40.001180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825c71-04a7-43b9-8c6d-50a3474e5d05 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:40.034461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:40.035103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:40.035392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:40.035673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:40.036492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] RBAC: Adding query filter params (role.id=a08f0a1b75f34a20a379690997b9cf81, scope.project.id=71e7caaaa0334938a02dd6939319e82c, user.id=3ca6592b46d6427098e2c764c9f6aac6) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:40.036808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:40.038251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904430c-81aa-47aa-84e8-177149be33c2 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:40.043961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 565/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:39 2022] GET /identity/v3/role_assignments?user.id=3ca6592b46d6427098e2c764c9f6aac6&scope.project.id=71e7caaaa0334938a02dd6939319e82c&role.id=a08f0a1b75f34a20a379690997b9cf81 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:42.222839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41485042-77d3-4b91-900c-f81d0c0d3d07 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:42.223409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41485042-77d3-4b91-900c-f81d0c0d3d07 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:42.223409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41485042-77d3-4b91-900c-f81d0c0d3d07 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:42.224773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 566/1131] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:42.229881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:42.230321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:42.230479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:42.247696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:42.248899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:42.313914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 566/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:42.337427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41485042-77d3-4b91-900c-f81d0c0d3d07 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:42.348607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:42.435405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41485042-77d3-4b91-900c-f81d0c0d3d07 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:42.467402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:42.468166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:42.468445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:42.468716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:42.469858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:42.471911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7e8e3-0368-4d46-8572-dfc96a62c0f7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:42.473538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 567/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:42.479239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:42.541597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:42.574568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cca47a-74c9-4199-993f-70e8ae87d307 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:42.607295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:42.607939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:42.608219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:42.608491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:42.609183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:42.610643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb347c2f-2d41-44c3-b71a-2cfaaba69ed0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:42.634668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 567/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:39:42 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:44.833772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1668b-a6b9-4748-8d5a-fc5b229ff26c None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:44.834339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1668b-a6b9-4748-8d5a-fc5b229ff26c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:44.834339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1668b-a6b9-4748-8d5a-fc5b229ff26c None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:44.835714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 568/1135] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:44.840868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:44.841319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:44.841422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:44.857790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:44.859044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:44.923613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 568/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:44.948192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1668b-a6b9-4748-8d5a-fc5b229ff26c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:45.022860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:45.046526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1668b-a6b9-4748-8d5a-fc5b229ff26c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:45.078673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:45.079441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:45.079719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:45.079987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:45.081141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:45.082625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba4f46-8f98-40eb-85b4-61cb37295925 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:45.084262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 569/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:39:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:45.089660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:45.137734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:45.187601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596479b-96b0-478b-bdf9-e6afee65706a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:45.221845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:45.222531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:45.222818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:45.223098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:45.223809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:45.225238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c2b6-2c49-4bed-b5dc-1f62ab31ed47 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:45.245720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 569/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:39:45 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:39:47.414982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3816-3446-4a1c-8422-5c1f2430907e None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:47.415498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3816-3446-4a1c-8422-5c1f2430907e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:47.415584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3816-3446-4a1c-8422-5c1f2430907e None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:47.416909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 570/1139] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:47.422191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:47.422561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:47.422743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:47.439598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:47.440794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:47.503872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 570/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:47.526758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3816-3446-4a1c-8422-5c1f2430907e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:47.624235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3816-3446-4a1c-8422-5c1f2430907e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:47.657081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:47.657855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:47.658171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:47.658451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:47.668825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:47.676467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:47.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:47.678669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 571/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:47.683428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:47.715584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:47.779356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80918-5030-4cb4-be65-469e84d79ff3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:47.811455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:47.812105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:47.812384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:47.812656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:47.813385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:47.813690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:47.815082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:47.822826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 571/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:47.828372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:47.832588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:47.926475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683d0c3-79f4-41f2-9ec4-2e2479afe4af None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:47.959584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:47.960435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:47.960725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:47.961015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] PATH_INFO: `/v3/users/dd473ca402224628951f3b6990fa31f4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:47.974477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] RBAC: Authorizing `identity:get_user(user_id=dd473ca402224628951f3b6990fa31f4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:47.975937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:47.977564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 572/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:47 2022] GET /identity/v3/users/dd473ca402224628951f3b6990fa31f4 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:47.983787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:48.075080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:48.080891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ccd3f-c006-41dd-94fe-1d30b317a8cc None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:48.112899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:48.113527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:48.113806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:48.114107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:48.179838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:48.181296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:48.269457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:48.276172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 572/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:48.281875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:48.339709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:48.376125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fccdf-1024-4e91-85f2-212ce96b9a52 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:48.408258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:48.409090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:48.409392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:48.409660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] PATH_INFO: `/v3/projects/75b4a027a9514a9fa6beb03e49882dcf` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:48.410838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] RBAC: Authorizing `identity:get_project(project_id=75b4a027a9514a9fa6beb03e49882dcf)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:48.412242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85b44-35d4-4024-92f8-4eb006fb3760 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:48.413894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 573/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:48 2022] GET /identity/v3/projects/75b4a027a9514a9fa6beb03e49882dcf => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:48.419780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:48.514382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dc8cc-14c4-4ce1-859f-eda92151d8ee None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:48.547297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:48.547926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:48.548205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:48.548476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:48.549296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=75b4a027a9514a9fa6beb03e49882dcf, user.id=dd473ca402224628951f3b6990fa31f4) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:48.549603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:48.551080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943e84b-3656-40a4-b51b-00af3122640e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:48.556282 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 573/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:48 2022] GET /identity/v3/role_assignments?user.id=dd473ca402224628951f3b6990fa31f4&scope.project.id=75b4a027a9514a9fa6beb03e49882dcf&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:50.688727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b164664-8c95-40e0-bab5-fcad1c107ad7 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:50.689275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b164664-8c95-40e0-bab5-fcad1c107ad7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:50.689275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b164664-8c95-40e0-bab5-fcad1c107ad7 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:50.691234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 574/1147] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:50.696446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:50.696844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:50.696997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:50.713201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:50.714417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:50.767732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:50.778959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 574/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:50.800718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b164664-8c95-40e0-bab5-fcad1c107ad7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:50.898258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b164664-8c95-40e0-bab5-fcad1c107ad7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:50.925968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:50.931815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:50.932586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:50.932869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:50.933145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:50.942976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:50.950421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:50.951558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:50.952582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 575/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:50.957288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:51.051681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17de06-697e-47d3-89e2-62bb05d88004 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:51.083214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:51.083862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:51.084141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:51.084411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:51.085138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:51.085442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:51.086832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:51.094514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 575/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:51.100856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:51.198214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8ca7a-be29-461d-9071-420f35c1d20f None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:51.231158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:51.232023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:51.232315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:51.232592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] PATH_INFO: `/v3/users/dd473ca402224628951f3b6990fa31f4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:51.233792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] RBAC: Authorizing `identity:get_user(user_id=dd473ca402224628951f3b6990fa31f4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:51.235259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:51.236899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 576/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:51 2022] GET /identity/v3/users/dd473ca402224628951f3b6990fa31f4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:51.242603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:51.294759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:51.341267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952e0eb-a336-4315-911c-a09d60637e50 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:51.373544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:51.374415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:51.374705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:51.374985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] PATH_INFO: `/v3/projects/75b4a027a9514a9fa6beb03e49882dcf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:51.376164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] RBAC: Authorizing `identity:get_project(project_id=75b4a027a9514a9fa6beb03e49882dcf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:51.377614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d9b65-9192-4807-8a7e-599ecd2744c7 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:51.379884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 576/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:51 2022] GET /identity/v3/projects/75b4a027a9514a9fa6beb03e49882dcf => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:51.385735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:51.442576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:51.482200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a0b8-8cf4-4f80-84b2-b34c6fbb79ed None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:51.514665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:51.515797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:51.516080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:51.516353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] PATH_INFO: `/v3/projects/75b4a027a9514a9fa6beb03e49882dcf/users/dd473ca402224628951f3b6990fa31f4/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:51.518304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] RBAC: Authorizing `identity:create_grant(project_id=75b4a027a9514a9fa6beb03e49882dcf, user_id=dd473ca402224628951f3b6990fa31f4, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:51.519751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70ba4e-88d5-436d-9f3b-93ecf2b0d816 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:51.531972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 577/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:39:51 2022] PUT /identity/v3/projects/75b4a027a9514a9fa6beb03e49882dcf/users/dd473ca402224628951f3b6990fa31f4/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:39:53.652836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45387fb7-ff6d-4aea-91a4-b2e5e38e0e49 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:53.653363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45387fb7-ff6d-4aea-91a4-b2e5e38e0e49 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:53.653475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45387fb7-ff6d-4aea-91a4-b2e5e38e0e49 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:53.654815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 577/1154] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:53.659887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:53.660314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:53.660455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:53.677577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:53.678813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:53.743304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 578/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:53.766423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45387fb7-ff6d-4aea-91a4-b2e5e38e0e49 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:53.865856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45387fb7-ff6d-4aea-91a4-b2e5e38e0e49 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:53.885564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:53.899605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:53.900386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:53.900671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:53.900946 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:53.911378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:39:53.918835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:53.919936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:39:53.920835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 578/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:53.926325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:53.972922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:54.023284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde84657-2711-485c-ad25-328f56cea5ca None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:54.057051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:54.057709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:54.058029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:54.058313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:54.059058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:54.059368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:54.060739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:54.068637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 579/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:54.075932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:54.173110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3206-85bb-458c-a4a0-1ab86bc227fe None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:54.205406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:54.206308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:54.206594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:54.206871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] PATH_INFO: `/v3/users/dd473ca402224628951f3b6990fa31f4` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:54.208041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] RBAC: Authorizing `identity:get_user(user_id=dd473ca402224628951f3b6990fa31f4)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:54.209472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:54.211103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 579/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:39:54 2022] GET /identity/v3/users/dd473ca402224628951f3b6990fa31f4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:54.216612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:54.314315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea7b41-7bdb-443a-b17b-2ee777bbc236 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:54.324186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:54.348218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:54.348857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:54.349143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:54.349421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:54.413976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:54.415430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:54.508593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 580/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:54.514369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:54.561930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:54.611142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761e3209-03b5-40f8-9f3a-b8110ac9ee37 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:54.644263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:54.645112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:54.645400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:54.645677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] PATH_INFO: `/v3/projects/75b4a027a9514a9fa6beb03e49882dcf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:54.646880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] RBAC: Authorizing `identity:get_project(project_id=75b4a027a9514a9fa6beb03e49882dcf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:54.648317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9022-8e5b-4362-a6df-2a6b54055184 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:54.650015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 580/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:39:54 2022] GET /identity/v3/projects/75b4a027a9514a9fa6beb03e49882dcf => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:54.655917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:54.667235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:54.751133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fdec3-fafa-4f1c-aa6d-97b1cf38f447 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:54.784602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:54.785188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:54.785473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:54.785752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:54.786613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=75b4a027a9514a9fa6beb03e49882dcf, user.id=dd473ca402224628951f3b6990fa31f4) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:54.786932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:54.788358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11f5cb-5919-40b0-84ca-37c442cb906c None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:54.794049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 581/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:39:54 2022] GET /identity/v3/role_assignments?user.id=dd473ca402224628951f3b6990fa31f4&scope.project.id=75b4a027a9514a9fa6beb03e49882dcf&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:56.967770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fa03-9200-4347-ae9f-c49e60d758b0 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:56.968354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fa03-9200-4347-ae9f-c49e60d758b0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:56.968354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7fa03-9200-4347-ae9f-c49e60d758b0 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:56.969679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 581/1162] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:56.975450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:56.975895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:56.976045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:56.993191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:56.994478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:57.058640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 582/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:57.080055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7fa03-9200-4347-ae9f-c49e60d758b0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:57.177798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7fa03-9200-4347-ae9f-c49e60d758b0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:57.193275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:57.211008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:57.211636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:57.211974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:57.212280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:57.278085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:57.279536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:57.373687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 582/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:39:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:39:57.379711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:57.382784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:57.475576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a47a01-2f5d-41c5-bb7e-0a6adb67e319 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:57.508365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:57.509099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:57.509380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:57.509652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:57.516412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:57.517853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-8669ccc2-30de-451a-aed0-77c88ccc2ed5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:39:57.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:39:57.533577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:39:57.534709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 583/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:39:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:39:57.539287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:39:57.556073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:39:57.635039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088d63-6df6-4544-b4f1-69dd7be91dc5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:39:57.668105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:39:57.668753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:57.669039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:57.669317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:57.670060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:39:57.670374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:39:57.671736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5361b-b62e-4b3d-959a-0a1ccb68029a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:39:57.678761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 583/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Mon May 23 10:39:57 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:39:59.829093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69cd04-5236-462d-9520-8865514421ff None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:59.829647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69cd04-5236-462d-9520-8865514421ff None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:59.829647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69cd04-5236-462d-9520-8865514421ff None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:59.831679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 584/1167] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:39:59.836811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0a1a-658b-4b08-b17a-70254b5564d7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:39:59.837245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0a1a-658b-4b08-b17a-70254b5564d7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:39:59.837354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0a1a-658b-4b08-b17a-70254b5564d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:39:59.853535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ec8d0a1a-658b-4b08-b17a-70254b5564d7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:39:59.854735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ec8d0a1a-658b-4b08-b17a-70254b5564d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:39:59.918314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 584/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:39:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:39:59.940634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69cd04-5236-462d-9520-8865514421ff None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:00.002985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:00.041455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69cd04-5236-462d-9520-8865514421ff None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:00.073958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:00.074634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:00.074918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:00.075196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:00.075942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:00.077315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b334d6-db4d-4b53-b691-3d49c9199ca0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:00.091296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 585/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Mon May 23 10:39:59 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:40:02.257809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e70b9-94a9-45ac-95d4-d9d819633cf8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:02.258263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e70b9-94a9-45ac-95d4-d9d819633cf8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:02.258399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e70b9-94a9-45ac-95d4-d9d819633cf8 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:02.259777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 585/1170] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:40:02.264893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:02.265573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:02.265710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:02.282767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:40:02.283941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:40:02.347979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 586/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:40:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:40:02.371254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e70b9-94a9-45ac-95d4-d9d819633cf8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:02.383893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:02.470087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e70b9-94a9-45ac-95d4-d9d819633cf8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:02.503280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:02.504073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:02.504360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:02.504637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] PATH_INFO: `/v3/domains/28196a8f8c424b419c67eae55598d072` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:02.514804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=28196a8f8c424b419c67eae55598d072)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:02.516521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54266df3-9a0a-4698-a7e4-d3b467ef90ee None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:02.518205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 586/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Mon May 23 10:40:02 2022] GET /identity/v3/domains/28196a8f8c424b419c67eae55598d072 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:02.523891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:02.576757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:02.620516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a6d2-a526-4b6a-a01d-c5cfe27dcdef None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:02.653016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:02.653664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:02.653961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:02.654253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:02.654952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:02.656373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac46b6-71da-48ab-93bc-4ea44155b3f9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:02.680771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 587/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:40:02 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:40:04.858058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98dfd1-c7d5-4fed-bf1c-e48639072661 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:04.858595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98dfd1-c7d5-4fed-bf1c-e48639072661 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:04.858595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98dfd1-c7d5-4fed-bf1c-e48639072661 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:04.859978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 587/1174] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:40:04.865626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:04.866144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:04.866244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:04.883101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:40:04.884266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:40:04.947876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 588/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:40:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:40:04.971593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98dfd1-c7d5-4fed-bf1c-e48639072661 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:05.040156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:05.071792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98dfd1-c7d5-4fed-bf1c-e48639072661 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:05.104111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:05.104962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:05.105246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:05.105523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] PATH_INFO: `/v3/domains/28196a8f8c424b419c67eae55598d072` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:05.106713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=28196a8f8c424b419c67eae55598d072)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:05.108199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86ca35-e478-4e3d-b181-60ae47d340e4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:05.109797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 588/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Mon May 23 10:40:04 2022] GET /identity/v3/domains/28196a8f8c424b419c67eae55598d072 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:05.116036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:05.149470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:05.211044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e28e-98e2-41ca-8559-43ce05d2cee6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:05.242839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:05.243502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:05.243783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:05.244055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:05.244767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:05.246266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c539ce2-406a-4724-890b-5e1d2797e7a7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:05.265619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 589/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:40:05 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:40:07.464132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ea0f-f1fc-40f5-8c00-48aefdc820bf None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:07.464677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ea0f-f1fc-40f5-8c00-48aefdc820bf None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:07.464677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ea0f-f1fc-40f5-8c00-48aefdc820bf None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:07.466091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 589/1178] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:40:07.471859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:07.472248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:07.472414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:07.489310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:40:07.490507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:40:07.553841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 590/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:40:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:40:07.576150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845ea0f-f1fc-40f5-8c00-48aefdc820bf None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:07.675425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845ea0f-f1fc-40f5-8c00-48aefdc820bf None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:07.708188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:07.708961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:07.709249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:07.709528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:07.716159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:07.720037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:40:07.727888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:40:07.728941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:40:07.729810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 590/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:40:07.734852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:07.758539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:07.831559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a8333-cd61-45d7-9494-0dbb98d7c8c5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:07.863402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:07.864050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:07.864330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:07.864604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:07.865335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:40:07.865653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:07.867047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:07.874738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 591/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:40:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:07.881064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:07.978646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02852a0-6e75-43a7-9dff-c4c13d413425 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:08.011324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:08.012178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:08.012465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:08.012743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] PATH_INFO: `/v3/users/413b1e3caae947d4a9167e3440f4840a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:08.025629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] RBAC: Authorizing `identity:get_user(user_id=413b1e3caae947d4a9167e3440f4840a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:08.027111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:08.028748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 591/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:40:07 2022] GET /identity/v3/users/413b1e3caae947d4a9167e3440f4840a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:08.034898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:08.116166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:08.131201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78e5e3-d86f-406c-9ffb-6039db51e739 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:08.162888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:08.163512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:08.163793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:08.164068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:08.227368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:08.228798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:08.307910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:08.323783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 592/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:40:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:40:08.329619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:08.383918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:08.427925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283757e-0f6b-425c-b452-8356d49b1bfa None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:08.460471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:08.461318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:08.461604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:08.461882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] PATH_INFO: `/v3/projects/36459de8a3384802a1925594d0111d0d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:08.463084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] RBAC: Authorizing `identity:get_project(project_id=36459de8a3384802a1925594d0111d0d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:08.464518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec6214-12e3-42e9-a33d-ebb79259c22e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:08.466238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 592/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:40:08 2022] GET /identity/v3/projects/36459de8a3384802a1925594d0111d0d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:08.472840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:08.566798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbc166-3ed7-418e-8207-5e7383d9ba2a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:08.599708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:08.600340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:08.600631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:08.600914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:08.601734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=36459de8a3384802a1925594d0111d0d, user.id=413b1e3caae947d4a9167e3440f4840a) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:40:08.602074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:08.603497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0bc4d-c342-4333-a926-6ded0b49dec0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:08.608735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 593/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:40:08 2022] GET /identity/v3/role_assignments?user.id=413b1e3caae947d4a9167e3440f4840a&scope.project.id=36459de8a3384802a1925594d0111d0d&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:10.730632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733c60-36e1-48ae-af2e-738372122852 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:10.731156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733c60-36e1-48ae-af2e-738372122852 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:10.731279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733c60-36e1-48ae-af2e-738372122852 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:10.732594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 593/1186] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:40:10.738271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d756-9543-46d9-a584-b7caa001562b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:10.738659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d756-9543-46d9-a584-b7caa001562b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:10.738825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396d756-9543-46d9-a584-b7caa001562b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:10.755612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b396d756-9543-46d9-a584-b7caa001562b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:40:10.756782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b396d756-9543-46d9-a584-b7caa001562b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:40:10.797881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:10.821054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 594/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:40:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:40:10.842941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733c60-36e1-48ae-af2e-738372122852 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:10.942655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733c60-36e1-48ae-af2e-738372122852 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:10.964865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:10.976101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:10.976877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:10.977161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:10.977433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:10.988328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:40:10.995847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:40:10.996801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:40:10.998003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 594/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:40:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:40:11.002865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396d756-9543-46d9-a584-b7caa001562b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:11.097132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396d756-9543-46d9-a584-b7caa001562b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:11.130125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:11.130776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:11.131058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:11.131332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:11.132064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:40:11.132368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:11.133734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:11.141726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 595/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:11.147354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:11.243740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2437a3-30b1-4193-98f0-1904dfea8dea None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:11.276959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:11.277824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:11.278144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:11.278423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] PATH_INFO: `/v3/users/413b1e3caae947d4a9167e3440f4840a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:11.279602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] RBAC: Authorizing `identity:get_user(user_id=413b1e3caae947d4a9167e3440f4840a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:11.281032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:11.282690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 595/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:40:11 2022] GET /identity/v3/users/413b1e3caae947d4a9167e3440f4840a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:11.288347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:11.335657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:11.392285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f539f-1cbe-4ba8-9d8f-353dac2dfa1e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:11.423871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:11.425244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:11.425530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:11.425807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] PATH_INFO: `/v3/projects/36459de8a3384802a1925594d0111d0d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:11.427047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=36459de8a3384802a1925594d0111d0d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:11.428470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8103093-7250-449b-bb66-e6a4c5aff1f7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:11.430188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 596/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:40:11 2022] GET /identity/v3/projects/36459de8a3384802a1925594d0111d0d => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:11.436184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:11.487704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:11.533253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11122a92-396b-477e-b007-d84b4e7cf061 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:11.565566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:11.566466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:11.566752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:11.567028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] PATH_INFO: `/v3/projects/36459de8a3384802a1925594d0111d0d/users/413b1e3caae947d4a9167e3440f4840a/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:11.568953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] RBAC: Authorizing `identity:create_grant(project_id=36459de8a3384802a1925594d0111d0d, user_id=413b1e3caae947d4a9167e3440f4840a, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:11.570421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a6544-1bff-49c6-9ae9-f440d90f7135 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:11.582346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 596/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:40:11 2022] PUT /identity/v3/projects/36459de8a3384802a1925594d0111d0d/users/413b1e3caae947d4a9167e3440f4840a/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:40:13.721486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd5a-32f1-4221-95c1-6b60a6b4419e None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:13.722092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd5a-32f1-4221-95c1-6b60a6b4419e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:13.722092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48dd5a-32f1-4221-95c1-6b60a6b4419e None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:13.723578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 597/1193] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:40:13.728904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:13.729294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:13.729468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:13.746635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:40:13.747835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:40:13.813211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 597/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:40:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:40:13.835917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48dd5a-32f1-4221-95c1-6b60a6b4419e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:13.933204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48dd5a-32f1-4221-95c1-6b60a6b4419e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:13.941549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:13.965121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:13.965891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:13.966202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:13.966477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:13.977071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:40:13.985084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:40:13.986213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:40:13.987900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 598/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:40:13.992893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:14.034613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:14.088644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781add1-4272-4142-8c3e-976ea1c6caff None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:14.121739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:14.122420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:14.122708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:14.122987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:14.123728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:40:14.124038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:14.125427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:14.133256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 598/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:14.138749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:14.234430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71497b35-e9ea-4ba5-b325-717bec96cb21 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:14.267394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70313871-f406-4003-9264-58e3418aac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:14.268895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313871-f406-4003-9264-58e3418aac18 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:14.269173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313871-f406-4003-9264-58e3418aac18 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:14.269444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70313871-f406-4003-9264-58e3418aac18 None admin] PATH_INFO: `/v3/users/413b1e3caae947d4a9167e3440f4840a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:14.270645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70313871-f406-4003-9264-58e3418aac18 None admin] RBAC: Authorizing `identity:get_user(user_id=413b1e3caae947d4a9167e3440f4840a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:14.272072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70313871-f406-4003-9264-58e3418aac18 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:14.273689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 599/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Mon May 23 10:40:14 2022] GET /identity/v3/users/413b1e3caae947d4a9167e3440f4840a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:14.279300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:14.375368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eed26e-51b7-45cd-ad72-c35605627647 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:14.378236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:14.408297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:14.408928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:14.409217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:14.409496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:14.474167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:14.475614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:14.570482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 599/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:40:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:40:14.576114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70313871-f406-4003-9264-58e3418aac18 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:14.609431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:14.671697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70313871-f406-4003-9264-58e3418aac18 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:14.703994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:14.704829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:14.705111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:14.705384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] PATH_INFO: `/v3/projects/36459de8a3384802a1925594d0111d0d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:14.706561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] RBAC: Authorizing `identity:get_project(project_id=36459de8a3384802a1925594d0111d0d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:14.707981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74db3b-954d-43cb-bd7f-de7a4517f8d8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:14.709646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 600/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Mon May 23 10:40:14 2022] GET /identity/v3/projects/36459de8a3384802a1925594d0111d0d => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:14.715476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:40:14.718528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:14.811659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9a32d-ea24-4a48-9c27-67e96cced231 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:40:14.843866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:40:14.844453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:40:14.844740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:40:14.845017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:40:14.846452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=36459de8a3384802a1925594d0111d0d, user.id=413b1e3caae947d4a9167e3440f4840a) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:40:14.846770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:40:14.848193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab9da6-72e0-496f-b466-75a5f24cc1c5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:40:14.853847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 600/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:40:14 2022] GET /identity/v3/role_assignments?user.id=413b1e3caae947d4a9167e3440f4840a&scope.project.id=36459de8a3384802a1925594d0111d0d&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:40:24.700033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=48 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:40:24.851265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=66 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:14.287292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9f7a-f5c4-4c81-89f1-861489b7bcf8 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:14.287931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9f7a-f5c4-4c81-89f1-861489b7bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:14.287931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9f7a-f5c4-4c81-89f1-861489b7bcf8 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:14.289204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 601/1201] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:14.294816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:14.295307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:14.295424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:14.315304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:14.316405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:14.408806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 601/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:14.433930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f9f7a-f5c4-4c81-89f1-861489b7bcf8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:14.559766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f9f7a-f5c4-4c81-89f1-861489b7bcf8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:14.598002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:14.598829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:14.599136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:14.599416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:14.612187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:14.613929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:14.641269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:14.642555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:14.643522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:14.643522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:14.643522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:14.643522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:14.643522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 602/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:14.650595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:14.781542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d6fab-b960-4c07-b3eb-86fdc0f23b89 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:14.818784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:14.826995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:14.827803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:14.828687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:14.828953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:14.829739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:14.830077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:14.833558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b40d9-0bf6-4a97-824c-2ad769fb5edb None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:14.853631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 602/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:14.854999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:14.919902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:14.983262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e4be6-a619-4213-98d7-d2c2782a76b4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:15.029978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:15.029978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:15.029978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:15.029978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:15.030464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:15.030464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80d854-8136-4309-ac37-46352ffe0a91 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:15.053766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 603/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Mon May 23 10:41:14 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:41:17.462660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f7052-89c1-4d70-9d17-b0fb1e4c8e00 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:17.463221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f7052-89c1-4d70-9d17-b0fb1e4c8e00 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:17.463316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f7052-89c1-4d70-9d17-b0fb1e4c8e00 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:17.464903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 603/1206] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:17.470667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:17.471080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:17.471266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:17.490571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:17.491760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:17.585991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 604/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:17.605713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f7052-89c1-4d70-9d17-b0fb1e4c8e00 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:17.731030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f7052-89c1-4d70-9d17-b0fb1e4c8e00 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:17.769144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:17.770002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:17.770303 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:17.770603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:17.781511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:17.789072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:17.790222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:17.791097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:17.794314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 604/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:41:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:17.796279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:17.844872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:17.918532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21a276-a9ba-45e7-9347-ecdc6bdcac9b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:17.963135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:17.963961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:17.964256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:17.964540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:17.966681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:17.967109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:17.968972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:17.981830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 605/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:41:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:17.990073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:18.059217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:18.130352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2237cc-952a-488d-a2cf-350272c2b40c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:18.175724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:18.176844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:18.177201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:18.177665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:18.194776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:18.196297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:18.217057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:18.218182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:18.220119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:18.220119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:18.220119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:18.220119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:18.220119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 605/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:18.225507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:18.352125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:18.356302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd26922-6169-4861-b515-22fdb2971a13 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:18.406972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:18.407933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:18.408384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:18.408843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:18.409589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:18.409904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:18.411355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:18.432297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 606/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:18.433836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:18.564082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8150c-8cd0-48c1-a992-d715f7035bf9 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:18.565815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:18.606806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:18.607653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:18.607940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:18.608217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:18.617158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:18.618912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:18.636064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:18.637369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:18.638449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 606/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:18.644392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:18.775070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b4fae-c59d-4e15-8e5d-73392a06acfd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:18.817126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:18.817969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:18.818365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:18.818742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:18.819969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:18.820363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:18.822389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:18.845114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 607/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:18.851142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:18.994432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980585fc-c946-4ce5-ae4d-e484684d1b53 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:19.037088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:19.037745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:19.038137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:19.038335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:19.081803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:19.134060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:19.136257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:19.260581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 607/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:41:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:41:19.267384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:19.289688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:19.377511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8800c9d-7d6c-41ec-a46c-4c7bbba1f6c1 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:19.414345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:19.415131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:19.415409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:19.415680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:19.426381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:19.427824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:19.448346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:19.450623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:19.451744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:19.451744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:19.451744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:19.451744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:19.451744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 608/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:19.459292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:19.537676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:19.582671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fef95-778b-478a-ab5e-b3303966b2ef None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:19.618707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:19.619640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:19.620088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:19.620526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:19.621404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:19.621867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:19.623458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:19.635593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 608/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:19.641911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:19.722120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:19.759135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64e9a8-0405-4b9f-81fb-fdab90045e75 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:19.802632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:19.803437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:19.803723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:19.804036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:19.811005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:19.812504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:19.826066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:19.827164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:19.828768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 609/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:19.834015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:19.953286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823298a1-7e83-4194-a2c4-822f9feb252c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:19.980233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:19.997135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:19.998127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:19.998627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:19.998950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:20.000597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:20.001012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:20.002641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f590a32-b3e9-4323-94a7-c824e0277a2c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:20.021556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 609/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:20.028776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:20.157401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a0fb4-c689-4f58-acd1-76e185e15cf9 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:20.194257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:20.203686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:20.204602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:20.204899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:20.205175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:20.206309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=c27854a0a16544cc9eab7c048fe03833) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:20.206604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:20.208286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb64749-9323-47a1-9b4a-4a308c9a70aa None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:20.218699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 610/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:41:20 2022] GET /identity/v3/role_assignments?user.id=c27854a0a16544cc9eab7c048fe03833&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:22.497850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6838d-f1fb-4b2b-b801-b3b0bffab72f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:22.498521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6838d-f1fb-4b2b-b801-b3b0bffab72f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:22.498521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6838d-f1fb-4b2b-b801-b3b0bffab72f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:22.500284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 610/1220] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:22.506260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:22.506713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:22.506836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:22.524544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:22.525764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:22.599586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 611/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:22.622800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6838d-f1fb-4b2b-b801-b3b0bffab72f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:22.731796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6838d-f1fb-4b2b-b801-b3b0bffab72f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:22.775793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:22.776671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:22.776977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:22.777253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:22.788920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:22.798182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:22.799565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:22.800417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 611/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:41:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:22.805702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:22.913798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861571ed-b2ba-4468-8e9f-4fb0be6db690 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:22.941729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:22.952735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:22.953408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:22.953697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:22.953999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:22.954748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] RBAC: Adding query filter params (name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:22.955063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:22.956479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:22.965500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 612/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:41:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:22.972220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:23.025510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:23.092714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142bc10-9404-439e-9a7c-82958c3daaaa None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:23.127954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:23.128980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:23.129266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:23.129589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:23.140799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:23.142303 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:23.159996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:23.161248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:23.162585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:23.162585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:23.162585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:23.162585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:23.162585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 612/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:41:23.167347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:23.281697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19869-b683-445c-adf4-5281f66fc178 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:23.317246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:23.317914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:23.318238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:23.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:23.321125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:23.321125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:23.321125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:23.333087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 613/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:23.339024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:23.442352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:23.449875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17cd07-2ccf-4718-895b-c0d9ab08307b None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:23.488769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:23.489593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:23.489884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:23.490267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:23.497968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:23.500151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:23.514799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:23.516367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:23.517326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 613/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:41:23.522445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:23.573456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:23.638816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb525c-c74f-40ea-ad8f-365f4a2a836e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:23.678928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:23.679838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:23.680154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:23.680452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:23.681321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:23.681631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:23.683185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:23.704450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 614/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:23.711757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:23.828449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeab30-275c-45dc-b0f5-2a5d4c595f39 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:23.877849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:23.878811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:23.879095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:23.879364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:23.892312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:23.894185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:23.913043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:23.914193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:23.915502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:23.915502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:23.915502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:23.915502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:23.915502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 614/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:23.920471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:24.034662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc17f86-bcb3-496f-81a3-6f1cd1b513ee None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:24.037565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:24.070257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:24.070919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:24.071200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:24.071470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:24.072192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:24.072502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:24.073872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:24.085597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 615/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:24.091347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:24.101664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:24.197146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90f0ad-65ce-40e2-bbf4-ea373a5ed1d8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:24.236521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:24.237355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:24.237649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:24.237938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:24.245130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:24.246650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:24.260934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:24.262265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:24.263506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 615/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:41:24.270096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:24.389073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ee072-76cd-4395-9c44-0115fba376f8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:24.431468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:24.432240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:24.432532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:24.432812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:24.433732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:24.434100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:24.435951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcc8f8-f5ab-4391-b0c7-799de1deb36f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:24.451051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 616/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:24.458594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:24.527504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:24.566039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01bfb-e458-4edb-a599-883de336f1e1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:24.605942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:24.607472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:24.607951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:24.608421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/c27854a0a16544cc9eab7c048fe03833/roles/4a098a865b974d5190b1c6b19f06d3a2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:24.631694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=c27854a0a16544cc9eab7c048fe03833, role_id=4a098a865b974d5190b1c6b19f06d3a2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:24.634346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84465f-4bb5-4346-ad12-74f536dab5cd None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:24.651480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 616/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:41:24 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/c27854a0a16544cc9eab7c048fe03833/roles/4a098a865b974d5190b1c6b19f06d3a2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:41:27.051650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1028da-90e3-4c01-8dce-e3928436c539 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.052210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1028da-90e3-4c01-8dce-e3928436c539 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.052210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1028da-90e3-4c01-8dce-e3928436c539 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.053577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 617/1233] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:27.058887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.059335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.059483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.079428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:27.080784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:27.162059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 617/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:27.186196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1028da-90e3-4c01-8dce-e3928436c539 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:27.212595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:27.297320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1028da-90e3-4c01-8dce-e3928436c539 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:27.336227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:27.337120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.337478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.337826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.350557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:27.358610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:27.359803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:27.360870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 618/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Mon May 23 10:41:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:41:27.368218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:27.416753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:27.479727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dd414-613d-4ab1-8145-2e1bdb4274a8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:27.519346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:27.520110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.520462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.520806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.521617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:27.522018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:27.523710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:27.537446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 618/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Mon May 23 10:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:27.538623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:27.645772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:27.666416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90508ab4-8ccb-48c7-821f-4fa504a4ac16 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:27.710259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:27.711501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.711614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.711982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.724047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:27.733348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:27.759169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:27.760185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:27.761144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:27.761144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:27.761144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:27.761144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:27.761144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 619/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:27.767448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:27.871057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32e4ff-5af0-4bce-b606-3d8c28ff93a4 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:27.880761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:27.909174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:27.909780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:27.910130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:27.910446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:27.911168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:27.911476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:27.913008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:27.926569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 619/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:27.932288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:28.044928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac535-83c4-415c-8aa4-7208decec3c7 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:28.083122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:28.083959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:28.084244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:28.084522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] PATH_INFO: `/v3/users/placement` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:28.087685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:28.093563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:28.095223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:28.111797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:28.113115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:28.114408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:28.118203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 620/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:28.119523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:28.228078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4c8e9-871c-4498-9bc7-57088649b062 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:28.269461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:28.270362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:28.270787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:28.271204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:28.272126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:28.272434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:28.273869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:28.294734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 620/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:28.301134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:28.402325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20505b-3fae-4376-aa74-92ad9c27218a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:28.439074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:28.439074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:28.439074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:28.439074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:28.510904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:28.519688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:28.521500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:28.626519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 621/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:41:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:41:28.633182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:28.669336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:28.754689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdad484-ea0b-407f-88e5-0a1eaa534e5c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:28.796515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:28.797636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:28.798008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:28.798358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:28.810182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:28.812111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:28.831606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:28.832785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:28.838549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:28.838549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:28.838549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:28.838549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:28.838549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 621/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:28.838951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:28.950253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:28.957374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb489af-6314-4d33-888a-0aea185dea1a None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:28.996294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:28.997101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:28.997405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:28.997851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:28.998723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:28.999843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:29.002293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:29.021594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 622/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:29.023525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:29.121923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:29.146538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876c875-672d-4ada-81f3-b7856bb996a5 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:29.185503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:29.186352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:29.187149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:29.187479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:29.194421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:29.195878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:29.210723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:29.211815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:29.213205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 622/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:29.217902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:29.323689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b61e54-6297-46ee-9ac9-e3c531265715 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:29.359696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:29.361940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:29.361940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:29.361940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:29.361940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:29.362306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:29.363584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6806518-1b3a-42af-b66c-106738311eec None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:29.376438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 623/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:29.382687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:29.492667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af01f8-305b-4149-bb71-7c68347399e1 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:29.530090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:29.530746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:29.531027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:29.531300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:29.532119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] RBAC: Adding query filter params (role.id=4a098a865b974d5190b1c6b19f06d3a2, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=c27854a0a16544cc9eab7c048fe03833) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:29.532426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:29.533969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad75ebf-077e-4d6c-a700-fb52588d8818 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:29.540696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 623/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:41:29 2022] GET /identity/v3/role_assignments?user.id=c27854a0a16544cc9eab7c048fe03833&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=4a098a865b974d5190b1c6b19f06d3a2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:31.904139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471103-1609-4d81-9262-7a19feab32bc None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:31.904733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471103-1609-4d81-9262-7a19feab32bc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:31.904733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0471103-1609-4d81-9262-7a19feab32bc None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:31.906595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 624/1247] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:31.912029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:31.912452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:31.912566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:31.917548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:31.931465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:31.932985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:32.016474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 624/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:32.043869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471103-1609-4d81-9262-7a19feab32bc None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:32.065509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:32.163295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0471103-1609-4d81-9262-7a19feab32bc None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:32.202889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:32.203679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:32.203958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:32.204238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:32.214952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:32.223058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:32.224133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:32.225394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 625/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:32.232222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:32.358458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa3831-9403-4180-9f78-0203afef5ed9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:32.393190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:32.399098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:32.399873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:32.400190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:32.400500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:32.401244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:32.401644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:32.403050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:32.412119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 625/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:32.419740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:32.506886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:32.532601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad13e41d-685f-477e-b4bd-12274d5d779e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:32.568122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:32.569014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:32.569325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:32.569609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:32.581641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:32.583177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:32.601159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:32.602363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:32.608140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:32.608140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:32.608140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:32.608140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:32.608140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 626/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:32.609029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:32.713411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f357d1-b6ee-4e62-91d7-1aac94bf0356 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:32.751130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:32.751818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:32.752101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:32.752375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:32.753087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:32.753393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:32.754810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:32.768217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 626/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:41:32.774053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:32.882226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3096e-b5de-4958-baa5-8dc1e3d09cbf None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:32.910701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:32.920478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:32.922052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:32.922454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:32.922673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:32.932130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:32.933609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:32.948135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:32.949151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:32.950533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 627/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:32.955430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:33.025390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:33.086266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87967045-41d2-4dac-b971-9975c04f86f3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:33.125664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:33.126467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:33.126753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:33.127030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:33.127877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:33.128202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:33.129638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:33.148340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 627/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:33.154405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:33.268099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cf89c-53d6-4310-ab4a-63a7152e3ae4 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:33.303134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:33.303804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:33.304090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:33.304368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:33.365176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:33.378319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:33.379805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:33.482450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 628/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:41:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:41:33.488453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:33.592275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea8a5f-05ec-4cab-9d12-c3be22d6bb7e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:33.606514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:33.628377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:33.629161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:33.629444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:33.629717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:33.640096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:33.641811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:33.659220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:33.660312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:33.661433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:33.661433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:33.661433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:33.661433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:33.661433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 628/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:33.666516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:33.759146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:33.772985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722868e6-5613-458c-a80e-dd4d343f60c2 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:33.810850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:33.811539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:33.811820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:33.812094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:33.812816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:33.813116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:33.814517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:33.826672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 629/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:33.832973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:33.932823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71533acf-f820-4d0d-9784-24b15b73e9a3 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:33.965148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:33.965922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:33.966227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:33.966498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:33.973459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:33.974935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:33.988108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:33.989360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:33.990634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 629/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:33.995511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:34.099595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e4a63-d5ef-41db-bde7-9d944d81dbbd None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:34.130923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:34.131566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:34.131850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:34.132125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:34.132897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:34.133205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:34.134652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229eea0c-0f23-4a5f-8e0f-cf146e604bdc None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:34.146919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 630/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:34.152927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:34.178456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:34.251224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8e903-2b37-4b94-9202-04e87888be39 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:34.284083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:34.284725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:34.285012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:34.285289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:34.286143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=c27854a0a16544cc9eab7c048fe03833) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:34.286460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:34.287915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d9b06-5e1f-40ce-b7cf-c0bb51061499 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:34.293213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 630/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:41:34 2022] GET /identity/v3/role_assignments?user.id=c27854a0a16544cc9eab7c048fe03833&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:36.432540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b681e-0452-41ba-a5eb-03176ef45998 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:36.433082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b681e-0452-41ba-a5eb-03176ef45998 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:36.433082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b681e-0452-41ba-a5eb-03176ef45998 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:36.434516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 631/1261] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:36.440070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:36.440485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:36.440631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:36.458404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:36.459593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:36.523768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 631/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:36.545079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b681e-0452-41ba-a5eb-03176ef45998 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:36.555691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:36.643252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b681e-0452-41ba-a5eb-03176ef45998 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:36.679353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:36.680460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:36.680885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:36.681310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:36.693705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:36.701274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:36.703039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:36.703936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 632/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:36.708882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:36.726477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:36.803742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb39ea-7454-48ca-8a1e-5189d779e041 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:36.835912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:36.836555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:36.836833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:36.837105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:36.837839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:36.838168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:36.839545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:36.847262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 632/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:36.852837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:36.916428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:36.949190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a43ed-4669-48ee-98b8-eee137e9522d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:36.980747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:36.981517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:36.981795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:36.982097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:36.993520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:36.995226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:37.014825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:37.016151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:37.017126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:37.017126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:37.017126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:37.017126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:37.017126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 633/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:37.022568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.109751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:37.128807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585d9a-dc7a-4ff9-bf32-9b1f6625754e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.160782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.161427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.161709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.162009 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.162717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:37.163021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.164381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.176097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 633/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:37.181782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.277676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d30b-add7-4aa1-b426-6faa22ff99ae None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.297113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:37.311063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.311853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.312132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.312401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] PATH_INFO: `/v3/users/placement` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.319577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.320995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:37.334665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:37.335766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:37.337153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 634/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:37.341571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.438922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd484a-7328-4242-bda8-dae5862e252e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.471341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.471990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.472271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.472547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.473368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:37.473702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.475157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.483940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:37.492194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 634/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:37.497581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.594254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b967-1802-44e4-9003-29c163d6542d None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.625558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.626360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.626645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.626919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.637078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.638547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:37.656316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:37.657331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:37.658442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:37.658442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:37.658442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:37.658442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:37.658442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 635/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 10:41:37.662975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.758653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245f5bf-4fe2-4541-bf7e-90a78af80235 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.792322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.792970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.793259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.793541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.794285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:37.794595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.796003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.807241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 635/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:37.812701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:37.844334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:37.909473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4901ed-8e32-4afc-a453-ce65235ac8a0 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:37.941165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:37.941932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:37.942243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:37.942517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:37.948893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:37.950358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:37.965142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:37.966191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:37.967156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 636/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:37.972464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:38.039176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:38.079224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd90818-50e2-4ede-9187-ac33aa8924d6 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:38.111600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:38.112242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:38.112523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:38.112796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:38.113573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:38.113877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:38.116137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f5a3c-aa80-49e7-8299-d5ff01e90bd0 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:38.127671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 636/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:38.133587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:38.222828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:38.231785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e297d-a4f7-4b6f-9bf4-b845dde010d8 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:38.265944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:38.266845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:38.267134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:38.267408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] PATH_INFO: `/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/c27854a0a16544cc9eab7c048fe03833/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:38.269358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] RBAC: Authorizing `identity:create_grant(project_id=52f7cf28d64e4d8f94b7da57a5afdcc8, user_id=c27854a0a16544cc9eab7c048fe03833, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:38.270836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09740e63-0dd0-44a0-8fcf-d3f3cf163bfc None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:38.282997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 637/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Mon May 23 10:41:38 2022] PUT /identity/v3/projects/52f7cf28d64e4d8f94b7da57a5afdcc8/users/c27854a0a16544cc9eab7c048fe03833/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:41:40.411475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7ee0-9c8f-444d-9182-348d820f2011 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:40.411990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7ee0-9c8f-444d-9182-348d820f2011 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:40.412109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7ee0-9c8f-444d-9182-348d820f2011 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:40.413405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 637/1274] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:40.418681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:40.419101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:40.419244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:40.436721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:40.437934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:40.501995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 638/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:40.525482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7ee0-9c8f-444d-9182-348d820f2011 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:40.624750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7ee0-9c8f-444d-9182-348d820f2011 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:40.641744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:40.658929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:40.659710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:40.659996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:40.660275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:40.670189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 10:41:40.678289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:40.679435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:40.680517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 638/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Mon May 23 10:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:40.685326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:40.763063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:40.782017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468477b0-4ec4-4975-88ae-eafe24f6946d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:40.815235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:40.815895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:40.816185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:40.816465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:40.817207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:40.817516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:40.818939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:40.827004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 639/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Mon May 23 10:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:40.832275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:40.929496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287f5a9-eba7-4e2e-9492-8b88fa4f20cd None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:40.961228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:40.962021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:40.962311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:40.962590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:40.973301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:40.974778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:40.992609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:40.993590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:40.994858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:40.994858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:40.994858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:40.994858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:40.994858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 639/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:40.999682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:41.109311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2d1e6-8e82-4141-97d8-46c86a8fae38 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:41.141059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:41.141714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:41.142025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:41.142305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:41.143019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:41.143326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:41.144687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:41.156464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 640/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:41.162084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:41.192709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:41.260344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0bf66-8a82-47d3-b9fa-841b4e429a0c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:41.292859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:41.293646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:41.293932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:41.294241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:41.302157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:41.303602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 10:41:41.317424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:41.318518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:41.319879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 640/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:41.324323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:41.328587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:41.420836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac85d75-b71d-4d18-a253-37c15b002e88 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:41.452993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:41.453650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:41.453934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:41.454240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:41.455068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:41.455376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:41.456790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:41.474352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 641/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Mon May 23 10:41:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:41.479931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:41.570183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:41.580399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39765413-85ea-4908-aff2-4b02c74c4b47 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:41.612588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:41.613222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:41.613508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:41.613789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:41.678108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:41.679509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:41.763336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:41.774223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 641/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Mon May 23 10:41:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 10:41:41.780122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:41.839072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:41.875603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545c73c-5045-4f16-a563-0caa1539128e None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:41.908034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:41.908802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:41.909080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:41.909350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:41.918918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:41.920356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 10:41:41.937481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:41.938548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:41.939680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 10:41:41.939680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 10:41:41.939680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 10:41:41.939680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:41.942102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 642/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:41.944358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:42.041014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a7fc7-72f5-402f-9b65-61032c153b8c None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:42.073714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:42.074381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:42.074662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:42.074934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:42.075640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:42.075943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:42.077301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:42.088008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 642/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Mon May 23 10:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:42.093551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:42.190900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66848b1e-6a91-4b31-a0e1-433aebe62707 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:42.204124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:42.223501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:42.224274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:42.224553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:42.224826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:42.231219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:42.232674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 10:41:42.246498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application with Lock( May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:42.247545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: ERROR keystone.server.flask.application May 23 10:41:42.248712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 643/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Mon May 23 10:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:42.254053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:42.301973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:42.349009 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1887400-657e-4aa4-a2ae-a53c9ccf7d5a None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:42.381234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:42.381872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:42.382186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:42.382459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:42.383225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:42.383528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:42.384942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953cfaaa-e958-4f09-9823-4ca074732e09 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:42.396622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 643/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Mon May 23 10:41:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:42.402632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:42.499115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af996e70-8555-4662-8ba6-fb0474344d69 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:42.531424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:42.532058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:42.532338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:42.532612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:42.533432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] RBAC: Adding query filter params (role.id=081e04042be64de0ac1c857efdae3324, scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8, user.id=c27854a0a16544cc9eab7c048fe03833) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:42.533741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:42.535187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be14fbaa-a30a-46a9-9350-9d2e5fd96f40 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:42.540775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 644/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Mon May 23 10:41:42 2022] GET /identity/v3/role_assignments?user.id=c27854a0a16544cc9eab7c048fe03833&scope.project.id=52f7cf28d64e4d8f94b7da57a5afdcc8&role.id=081e04042be64de0ac1c857efdae3324 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:44.723943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51894d-fa7e-4133-bc1b-622a1e6f621a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:44.724506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51894d-fa7e-4133-bc1b-622a1e6f621a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:44.724506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51894d-fa7e-4133-bc1b-622a1e6f621a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:44.725887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 644/1288] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:44.731147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:44.731568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:44.731712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:44.748220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:44.749944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:44.779135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:44.815761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 645/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:44.838205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51894d-fa7e-4133-bc1b-622a1e6f621a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:44.936766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51894d-fa7e-4133-bc1b-622a1e6f621a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:44.946955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:44.970431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:44.971208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:44.971496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:44.971776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:44.972418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:44.973789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:44.985493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:44.986722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:44.988062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 645/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:41:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:44.993187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:45.101166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e800953-c45b-4773-aee1-66af1176c7ca None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:45.134414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:45.135073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:45.135360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:45.135636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:45.136338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:45.136649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:45.138051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7864f-9c56-45ec-b10e-f894637c2151 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:45.143099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 646/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 23 10:41:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:45.148777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:45.245524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40895a22-e4d5-4a70-9856-bfcd2a88dc06 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:45.277995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:45.278644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:45.278926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:45.279199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] PATH_INFO: `/v3/services` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:45.279915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] RBAC: Adding query filter params (type=placement) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:45.280218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:45.281586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b7be8a-29d8-4d40-9c9c-528cb5f8758d None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:45.285897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 646/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 23 10:41:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:47.419836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d7f4-2b96-483e-900d-95b2491cc95f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:47.420853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d7f4-2b96-483e-900d-95b2491cc95f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:47.420948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d7f4-2b96-483e-900d-95b2491cc95f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:47.422302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 647/1293] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:47.427525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abcb67-3d8b-4d58-aa5b-faa5ae24d752 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:47.427961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abcb67-3d8b-4d58-aa5b-faa5ae24d752 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:47.428092 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abcb67-3d8b-4d58-aa5b-faa5ae24d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:47.445072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-56abcb67-3d8b-4d58-aa5b-faa5ae24d752 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:47.446312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-56abcb67-3d8b-4d58-aa5b-faa5ae24d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:47.474792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:47.511807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 647/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:41:47.534000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294d7f4-2b96-483e-900d-95b2491cc95f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:47.564713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:47.632445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294d7f4-2b96-483e-900d-95b2491cc95f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:47.664651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:47.665291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:47.665574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:47.665843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:47.666586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:47.667942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed89e85-0c34-4791-908f-f2400553697e None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:47.680554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 648/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Mon May 23 10:41:47 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:41:49.864218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa096e-b063-4b43-821c-715b1f6c1ede None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:49.864818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa096e-b063-4b43-821c-715b1f6c1ede None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:49.864818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa096e-b063-4b43-821c-715b1f6c1ede None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:49.866192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 648/1296] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:49.871329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:49.871703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:49.871882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:49.888839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:49.890118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:49.959290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 649/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 10:41:49.982014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa096e-b063-4b43-821c-715b1f6c1ede None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:50.080482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa096e-b063-4b43-821c-715b1f6c1ede None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:50.113354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:50.114162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:50.114492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:50.114720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] PATH_INFO: `/v3/services/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:50.115367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:50.116741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:50.128306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:50.129469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:50.130565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 649/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:41:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:50.135385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:50.153149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:50.231259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e4dbf-195c-48b2-aae6-4771b120a78c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:50.264682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:50.265611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:50.265895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:50.266204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:50.266916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:50.267226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:50.268590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:50.273660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 650/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 23 10:41:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:50.279496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:50.310596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:50.376357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d43201-117c-4c5c-a8a2-b4924f65c42e None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:50.409230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:50.409867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:50.410177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:50.410456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:50.411226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] RBAC: Adding query filter params (interface=public, service_id=c97ace02c7e74f3dbf960154289b6d59, region_id=RegionOne) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:50.411537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:50.412892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbcd2ef-6852-402b-a536-297b5247de73 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:50.417752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 650/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Mon May 23 10:41:50 2022] GET /identity/v3/endpoints?service_id=c97ace02c7e74f3dbf960154289b6d59&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:50.423351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:50.501546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:50.520597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fafbe7-cd0c-4514-b090-ed6e9afaba65 None admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:50.553320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:50.553997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:50.554279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:50.554551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:50.555224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:50.556566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ec0dd-5d9f-4856-818a-b7fef7c6a8d3 None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:50.563364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 651/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Mon May 23 10:41:50 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:41:52.724987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9425b-a218-4948-9b4f-393e3ea453b4 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:52.725514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9425b-a218-4948-9b4f-393e3ea453b4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:52.725514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9425b-a218-4948-9b4f-393e3ea453b4 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:52.726939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 651/1302] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:41:52.731980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:52.732385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:52.732533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:52.748934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:41:52.750140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:41:52.814300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 652/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:41:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 10:41:52.836521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9425b-a218-4948-9b4f-393e3ea453b4 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:52.928345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:52.937206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9425b-a218-4948-9b4f-393e3ea453b4 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:52.969488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:52.970290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:52.970574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:52.970849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:52.971484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:52.972837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 10:41:52.984226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 10:41:52.985316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.get_or_create( May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application with Lock( May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self._enter() May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application return self.creator() May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application created_value = creator( May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: ERROR keystone.server.flask.application May 23 10:41:52.986210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 652/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Mon May 23 10:41:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 10:41:52.992242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:53.075765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:41:53.098531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5cbb1-775d-4533-8845-a03d89d3f063 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:53.130963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:53.131616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:53.131902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:53.132179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] PATH_INFO: `/v3/services` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:53.132881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] RBAC: Adding query filter params (name=placement) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:41:53.133185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:53.134579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d2a94-b5ac-41a4-a446-f32a2ae22e6f None admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:53.140498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 653/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Mon May 23 10:41:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:41:53.146226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:41:53.242765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3c6c7-d79c-4f08-81fe-bfd5a52052c8 None admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:41:53.275849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:41:53.276482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:41:53.276764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:41:53.277040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:41:53.277771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:41:53.279159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad09e9a-7588-4639-9923-e841b1c0452b None admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:41:53.297057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 653/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Mon May 23 10:41:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:42:03.136552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=30 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:42:03.292281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:42:56.418469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca2f7-5f89-4453-bc36-c3900a329e8e None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:56.418902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca2f7-5f89-4453-bc36-c3900a329e8e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:56.419263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ca2f7-5f89-4453-bc36-c3900a329e8e None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:56.425682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26d3d-9ea9-4290-b3ef-08521792a971 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:56.426200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26d3d-9ea9-4290-b3ef-08521792a971 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:56.426715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c26d3d-9ea9-4290-b3ef-08521792a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:56.429377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 654/1307] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:42:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:42:56.506709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-23c26d3d-9ea9-4290-b3ef-08521792a971 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:42:56.508167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-23c26d3d-9ea9-4290-b3ef-08521792a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:42:56.630873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 654/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:42:56.681185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:56.681471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:56.681748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:56.683196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 655/1309] 10.222.0.36 () {54 vars in 964 bytes} [Mon May 23 10:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:42:56.690058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3296dd-4d0d-41e3-8add-aaa5fd7e2534 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:56.690354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3296dd-4d0d-41e3-8add-aaa5fd7e2534 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:56.690643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3296dd-4d0d-41e3-8add-aaa5fd7e2534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:56.774121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-da3296dd-4d0d-41e3-8add-aaa5fd7e2534 None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:42:56.775182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-da3296dd-4d0d-41e3-8add-aaa5fd7e2534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:42:56.797115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:42:56.854399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 655/1310] 10.222.0.36 () {58 vars in 1061 bytes} [Mon May 23 10:42:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:42:56.863834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:42:56.865546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:42:56.984896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de34945-5c1a-4d88-9aeb-a0bbfe7596c8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:42:57.062133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:42:57.062786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:57.063090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:57.063363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:57.065078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:42:57.121906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:42:57.180206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:42:57.182126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad000b-7b2a-4716-b0bc-97f6471e2235 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:42:57.201884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 656/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:42:59.670842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:59.671333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:59.671442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:59.672774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 656/1312] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:42:59.678622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac69d-5ec3-4b11-9ba3-5af6e1ed4993 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:42:59.679094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac69d-5ec3-4b11-9ba3-5af6e1ed4993 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:42:59.679892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ac69d-5ec3-4b11-9ba3-5af6e1ed4993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:42:59.774250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-cf2ac69d-5ec3-4b11-9ba3-5af6e1ed4993 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:42:59.775461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-cf2ac69d-5ec3-4b11-9ba3-5af6e1ed4993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:42:59.874602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 657/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:42:59.911756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:42:59.913305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:00.022782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b991-ed2a-41d9-b56d-ed210592f9f7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:00.093692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:00.096337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:00.097221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:00.097707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:00.098168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:00.099812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:00.206664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:00.207996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e49694-c05c-453a-ad3c-79a8d6c5eedb service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:00.225913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 657/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:02.733356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725f7e5-e860-4a28-bdaf-3acd76232024 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:02.733933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725f7e5-e860-4a28-bdaf-3acd76232024 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:02.733933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725f7e5-e860-4a28-bdaf-3acd76232024 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:02.735451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 658/1315] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:02.740635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b183c-ca3a-463a-ab35-55d00a3e94fc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:02.741025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b183c-ca3a-463a-ab35-55d00a3e94fc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:02.741198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b183c-ca3a-463a-ab35-55d00a3e94fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:02.815809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-be3b183c-ca3a-463a-ab35-55d00a3e94fc None None] MFA Rules not processed for user `55f99bba48794e39be6d96b2d2956952`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:02.817430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-be3b183c-ca3a-463a-ab35-55d00a3e94fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:02.874477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:02.922737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 658/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:02.979244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:02.979537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:02.979813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:02.981840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 659/1317] 10.222.0.36 () {54 vars in 964 bytes} [Mon May 23 10:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:02.988272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c37124-b0f2-4cd3-9558-561f91e736b4 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:02.988471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c37124-b0f2-4cd3-9558-561f91e736b4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:02.988710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c37124-b0f2-4cd3-9558-561f91e736b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:03.064028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-28c37124-b0f2-4cd3-9558-561f91e736b4 None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:03.065235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-28c37124-b0f2-4cd3-9558-561f91e736b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:03.140874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 659/1318] 10.222.0.36 () {58 vars in 1061 bytes} [Mon May 23 10:43:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:03.149816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:03.151343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:03.161488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:03.257042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f5c8-d051-4b50-aefc-cda4c8281c27 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:03.329903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:03.330574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:03.330854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:03.331124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:03.332740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:03.397388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:03.446779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:03.448260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeebb7e-52fb-4216-ac03-81093787ef71 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:03.466463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 660/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:06.540322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:06.540847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:06.540944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:06.542933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 660/1320] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:06.548082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5208-2f0f-4f48-8e78-4076e934c2d0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:06.548478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5208-2f0f-4f48-8e78-4076e934c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:06.548642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e5208-2f0f-4f48-8e78-4076e934c2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:06.611828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ea4e5208-2f0f-4f48-8e78-4076e934c2d0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:06.613045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ea4e5208-2f0f-4f48-8e78-4076e934c2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:06.707416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 661/1321] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:06.744376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:06.746536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:06.796879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:06.848612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9389d9-9368-478c-a53d-6e4369becd2a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:06.919417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:06.920053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:06.920336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:06.920612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:06.922263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:07.016713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:07.029914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:07.031397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b0c64-0387-47ab-a724-3b5c40a9926f service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:07.049365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 661/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:09.898546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:09.899083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:09.899177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:09.900529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 662/1323] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:09.906328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea172325-0a8d-4953-8e90-307670c83ade None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:09.906796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea172325-0a8d-4953-8e90-307670c83ade None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:09.906895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea172325-0a8d-4953-8e90-307670c83ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:09.971086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ea172325-0a8d-4953-8e90-307670c83ade None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:09.972291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ea172325-0a8d-4953-8e90-307670c83ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:10.073106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 662/1324] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:10.109119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:10.110669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:10.129599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:10.217645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c000f2c-9173-432c-9747-3c2cca22fd2e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:10.291553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:10.292207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:10.292489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:10.292767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:10.294421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:10.364513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:10.405072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:10.406608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2f67-adfa-4a00-873c-f67343f1727b service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:10.425173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 663/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:12.666741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:12.667279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:12.667372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:12.668719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 663/1326] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:12.674312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7b1d6-8712-40e1-aa14-f3c14dc88cd0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:12.674797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7b1d6-8712-40e1-aa14-f3c14dc88cd0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:12.674904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7b1d6-8712-40e1-aa14-f3c14dc88cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:12.738232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-73e7b1d6-8712-40e1-aa14-f3c14dc88cd0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:12.739422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-73e7b1d6-8712-40e1-aa14-f3c14dc88cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:12.836879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 664/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:12.873446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:12.875076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:12.935179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:12.979317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05ad34-df47-44ba-acaa-5327accedbe8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:13.053046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:13.053684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:13.053979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:13.054258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:13.055870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:13.160098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:13.162798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:13.164442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8da77-900f-4a0c-9e9b-1024e9f594b4 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:13.183264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 664/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:15.459834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:15.460360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:15.460449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:15.461779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 665/1329] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:15.467676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4b4d-e022-4cda-90b0-3561a241d3e4 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:15.468109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4b4d-e022-4cda-90b0-3561a241d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:15.468215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c4b4d-e022-4cda-90b0-3561a241d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:15.531791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-866c4b4d-e022-4cda-90b0-3561a241d3e4 None None] MFA Rules not processed for user `55f99bba48794e39be6d96b2d2956952`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:15.532914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-866c4b4d-e022-4cda-90b0-3561a241d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:15.631422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 665/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:15.668764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:15.670423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:15.701086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:15.773625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fa88d-97c1-4c0a-bd8a-c68b88ea3b52 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:15.848441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:15.849191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:15.849538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:15.849880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:15.851620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:15.928452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:15.960033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:15.961502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acde38b-ec41-42f0-8a98-b0e82a4ddb03 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:15.979567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 666/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:18.384269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:18.384708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:18.384832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:18.386439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 666/1332] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:18.391850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad0fcc-2e0d-4955-b006-afe31c05c6c2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:18.392268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad0fcc-2e0d-4955-b006-afe31c05c6c2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:18.392412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad0fcc-2e0d-4955-b006-afe31c05c6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:18.457000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-0cad0fcc-2e0d-4955-b006-afe31c05c6c2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:18.458256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0cad0fcc-2e0d-4955-b006-afe31c05c6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:18.556653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 667/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:43:18.592156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:18.593884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:18.672380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:18.707859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157e556-3d1c-4204-b397-c5917cb02ee8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:18.781002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:18.781657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:18.781970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:18.782254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:18.783926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:18.901036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:18.903145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bac29-27b1-4ec0-a75b-2ae4b4b1c58d service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:18.914340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:18.923755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 667/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:21.171772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:21.173577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:21.173577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:21.173784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 668/1335] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:21.178920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660339-bcb7-46b7-b143-bca8ae1bc7ce None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:21.179377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660339-bcb7-46b7-b143-bca8ae1bc7ce None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:21.179487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2660339-bcb7-46b7-b143-bca8ae1bc7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:21.242706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a2660339-bcb7-46b7-b143-bca8ae1bc7ce None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:21.243902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a2660339-bcb7-46b7-b143-bca8ae1bc7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:21.337405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 668/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:21.377261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:21.378845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:21.418885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:21.481183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca5509-8a7e-40a7-94de-0171bd1735ad None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:21.551146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:21.551784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:21.552091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:21.552364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:21.554013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:21.634667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:21.660258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:21.661675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e082a208-595a-48b7-a8b0-23e04d235bfd service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:21.679097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 669/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:24.177503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:24.178099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:24.178099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:24.179486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 669/1338] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:24.184780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c9372-d5a9-4e38-947b-c17d6ae70903 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:24.185228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c9372-d5a9-4e38-947b-c17d6ae70903 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:24.185333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c9372-d5a9-4e38-947b-c17d6ae70903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:24.248851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-634c9372-d5a9-4e38-947b-c17d6ae70903 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:24.250101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-634c9372-d5a9-4e38-947b-c17d6ae70903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:24.344325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 670/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:24.384567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:24.386700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:24.467448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:24.494734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f922cb-005d-40eb-906e-d915b5bfca74 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:24.566206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:24.566841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:24.567127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:24.567404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:24.569021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:24.674461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:24.675913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffdae0-bbbb-46d9-be69-75b8f1e4d43d service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:24.693990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 670/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:28.797921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d530-9496-4159-9931-9842346a30e4 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:28.798491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d530-9496-4159-9931-9842346a30e4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:28.798491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d530-9496-4159-9931-9842346a30e4 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:28.799910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 671/1341] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:28.805020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c914a84-f6bf-496e-8b0d-d5fa4b34bf25 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:28.805434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c914a84-f6bf-496e-8b0d-d5fa4b34bf25 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:28.805582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c914a84-f6bf-496e-8b0d-d5fa4b34bf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:28.812430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:28.878483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8c914a84-f6bf-496e-8b0d-d5fa4b34bf25 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:28.879700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8c914a84-f6bf-496e-8b0d-d5fa4b34bf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:28.985006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 671/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:29.024896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d530-9496-4159-9931-9842346a30e4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:29.027639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a902d530-9496-4159-9931-9842346a30e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:29.077649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:29.123884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d530-9496-4159-9931-9842346a30e4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:29.127472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:29.128119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:29.128400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:29.128675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:29.138086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:29.139531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a61aeb-83e4-411c-ba44-a3105c7b1a48 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:29.149240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 672/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:31.971094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf936-f75c-4837-8c59-743e5296b4ce None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:31.971651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf936-f75c-4837-8c59-743e5296b4ce None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:31.971651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bf936-f75c-4837-8c59-743e5296b4ce None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:31.973041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 672/1344] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:31.978390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b3dc-4537-4673-93cd-4b9f86f692b9 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:31.978800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b3dc-4537-4673-93cd-4b9f86f692b9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:31.979095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b3dc-4537-4673-93cd-4b9f86f692b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:32.008435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-0d81b3dc-4537-4673-93cd-4b9f86f692b9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:32.009803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0d81b3dc-4537-4673-93cd-4b9f86f692b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:32.093228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 673/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:32.129443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf936-f75c-4837-8c59-743e5296b4ce None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:32.139942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bf936-f75c-4837-8c59-743e5296b4ce None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:32.143674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:32.144300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:32.144580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:32.144852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:32.154252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:32.155681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984012-d8f5-4373-b5e9-12fef9686248 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:32.165154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 673/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:36.268072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74593-090f-4457-9479-52079091c6ab None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:36.268591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74593-090f-4457-9479-52079091c6ab None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:36.268758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b74593-090f-4457-9479-52079091c6ab None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:36.270174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 674/1347] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:36.275403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dccc1b-cfc4-4579-a257-60e1a43ff905 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:36.275826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dccc1b-cfc4-4579-a257-60e1a43ff905 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:36.276578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dccc1b-cfc4-4579-a257-60e1a43ff905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:36.294423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-97dccc1b-cfc4-4579-a257-60e1a43ff905 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:36.297881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-97dccc1b-cfc4-4579-a257-60e1a43ff905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:36.305463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 674/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:36.343445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b74593-090f-4457-9479-52079091c6ab None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:36.345145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-51b74593-090f-4457-9479-52079091c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:36.359197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b74593-090f-4457-9479-52079091c6ab None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:36.362766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:36.363412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:36.363690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:36.363965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:36.376024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:36.377522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d425326-3edc-4b0f-85b7-e58ea45a15fc service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:36.388459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 675/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:41.759881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f44db-fab1-45df-bf7d-5ed3b9778186 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:41.760969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f44db-fab1-45df-bf7d-5ed3b9778186 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:41.761212 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f44db-fab1-45df-bf7d-5ed3b9778186 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:41.763143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 675/1350] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:41.768565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f015-dd8c-4351-a80f-5ca646c5b397 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:41.769000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f015-dd8c-4351-a80f-5ca646c5b397 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:41.769174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0f015-dd8c-4351-a80f-5ca646c5b397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:41.841835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-65c0f015-dd8c-4351-a80f-5ca646c5b397 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:41.843000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-65c0f015-dd8c-4351-a80f-5ca646c5b397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:41.871161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:41.924464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 676/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:43:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:46.289379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=72 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:43:46.292516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496283-1bde-4fb7-8ba1-30a2de0b3841 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:46.293200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496283-1bde-4fb7-8ba1-30a2de0b3841 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:46.293979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d496283-1bde-4fb7-8ba1-30a2de0b3841 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:46.295838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 676/1352] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:46.301222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5259bf-2fc9-4019-bebd-877ec212db12 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:46.301700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5259bf-2fc9-4019-bebd-877ec212db12 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:46.301804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5259bf-2fc9-4019-bebd-877ec212db12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:46.322100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-eb5259bf-2fc9-4019-bebd-877ec212db12 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:46.324049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-eb5259bf-2fc9-4019-bebd-877ec212db12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:46.333346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 677/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:43:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:47.753451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a238-324d-4451-9642-c13b2e6c9348 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:47.754077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a238-324d-4451-9642-c13b2e6c9348 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:47.754077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a238-324d-4451-9642-c13b2e6c9348 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:47.755565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 677/1354] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:47.760862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435563-3a84-4806-abd2-1b1821db714d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:47.761264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435563-3a84-4806-abd2-1b1821db714d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:47.761548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78435563-3a84-4806-abd2-1b1821db714d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:47.779892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-78435563-3a84-4806-abd2-1b1821db714d None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:47.781191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-78435563-3a84-4806-abd2-1b1821db714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:47.789915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 678/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:43:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:51.592889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185e67-98a5-41b3-8538-85e5c1d66af6 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:51.593473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185e67-98a5-41b3-8538-85e5c1d66af6 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:51.593473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb185e67-98a5-41b3-8538-85e5c1d66af6 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:51.594837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 678/1356] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:51.599979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd7d8-c124-441d-884a-617fcc9eb1e8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:51.600766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd7d8-c124-441d-884a-617fcc9eb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:51.601041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcd7d8-c124-441d-884a-617fcc9eb1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:51.618635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bdbcd7d8-c124-441d-884a-617fcc9eb1e8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:51.620361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bdbcd7d8-c124-441d-884a-617fcc9eb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:51.628273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 679/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:51.683134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c1183-dd3e-4001-8a28-266ec1c1e0c3 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:51.683788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c1183-dd3e-4001-8a28-266ec1c1e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:51.684399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c1183-dd3e-4001-8a28-266ec1c1e0c3 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:51.686394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 679/1358] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 23 10:43:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:51.695312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365552-389b-4fc0-9277-b9e846df7d8a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:51.696170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365552-389b-4fc0-9277-b9e846df7d8a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:51.698070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8365552-389b-4fc0-9277-b9e846df7d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:51.747545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c8365552-389b-4fc0-9277-b9e846df7d8a None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:51.748707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c8365552-389b-4fc0-9277-b9e846df7d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:51.834754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 680/1359] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 23 10:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:51.842310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c1183-dd3e-4001-8a28-266ec1c1e0c3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:51.856795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c1183-dd3e-4001-8a28-266ec1c1e0c3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:51.862198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:51.862850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:51.863134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:51.863418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:51.872180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:51.873775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51d4be-7c55-412e-8a57-794124c32c83 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:51.883492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 680/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:54.277595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85982d-3074-4739-8b0f-bd38af01c7a4 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:54.278230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85982d-3074-4739-8b0f-bd38af01c7a4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:54.278230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85982d-3074-4739-8b0f-bd38af01c7a4 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:54.279691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 681/1361] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:54.284729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564dd6-5e18-4f86-9d65-b776d02d6b57 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:54.285139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564dd6-5e18-4f86-9d65-b776d02d6b57 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:54.285281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad564dd6-5e18-4f86-9d65-b776d02d6b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:54.305273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ad564dd6-5e18-4f86-9d65-b776d02d6b57 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:54.306502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ad564dd6-5e18-4f86-9d65-b776d02d6b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:54.314493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 681/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:54.358517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85982d-3074-4739-8b0f-bd38af01c7a4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:54.373375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85982d-3074-4739-8b0f-bd38af01c7a4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:54.379025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:54.379841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:54.380194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:54.380565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:54.390149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:54.392009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3edb8a-9a9d-4df1-986f-704b0d426e5f service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:54.403151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 682/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:56.937650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca171a-7fd3-49c9-8f8d-c596997fac3e None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:56.938184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca171a-7fd3-49c9-8f8d-c596997fac3e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:56.938280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca171a-7fd3-49c9-8f8d-c596997fac3e None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:56.939620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 682/1364] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:43:56.944854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855436a-d7c5-4fb4-9fbf-d3d32bc9fe79 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:56.945287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855436a-d7c5-4fb4-9fbf-d3d32bc9fe79 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:56.945420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855436a-d7c5-4fb4-9fbf-d3d32bc9fe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:56.964037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2855436a-d7c5-4fb4-9fbf-d3d32bc9fe79 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:56.965274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2855436a-d7c5-4fb4-9fbf-d3d32bc9fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:56.973090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 683/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:57.019235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d9b2d-6774-4fdb-95e4-8d4af1dbf31e None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:57.019594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d9b2d-6774-4fdb-95e4-8d4af1dbf31e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:57.019942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d9b2d-6774-4fdb-95e4-8d4af1dbf31e None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:57.021395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 683/1366] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 23 10:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:57.026623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24d5a-c025-42fd-a779-cad421c58bac None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:57.026913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24d5a-c025-42fd-a779-cad421c58bac None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:57.027193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc24d5a-c025-42fd-a779-cad421c58bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:57.058812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5dc24d5a-c025-42fd-a779-cad421c58bac None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:57.060033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5dc24d5a-c025-42fd-a779-cad421c58bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:57.069347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 684/1367] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 23 10:43:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:43:57.077199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d9b2d-6774-4fdb-95e4-8d4af1dbf31e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:57.106234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d9b2d-6774-4fdb-95e4-8d4af1dbf31e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:57.116176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:57.117102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:57.117388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:57.117666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:57.127503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:57.129090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705d552-87a0-430d-916e-fc51aee4134b service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:57.138721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 684/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:59.411160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbe73-223d-4c64-9526-7bc552634cd7 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:59.411694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbe73-223d-4c64-9526-7bc552634cd7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:59.411694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fbe73-223d-4c64-9526-7bc552634cd7 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:59.413017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 685/1369] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:59.418627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc8b-7016-4c98-883c-4183c0d887b3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:59.419423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc8b-7016-4c98-883c-4183c0d887b3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:59.419710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cc8b-7016-4c98-883c-4183c0d887b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:59.438202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-99f3cc8b-7016-4c98-883c-4183c0d887b3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:59.439419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-99f3cc8b-7016-4c98-883c-4183c0d887b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:59.450086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 685/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:43:59.492555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fbe73-223d-4c64-9526-7bc552634cd7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:43:59.508667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fbe73-223d-4c64-9526-7bc552634cd7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:43:59.514845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:43:59.515499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:59.515805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:59.516355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:59.525203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:43:59.527067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8971d2-2415-418a-9245-acc8b48c7f0b service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:43:59.537075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 686/1371] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:43:59.795291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975354-0dc0-4b86-b5e6-a0b7eacbc53a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:59.795578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975354-0dc0-4b86-b5e6-a0b7eacbc53a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:59.796040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c975354-0dc0-4b86-b5e6-a0b7eacbc53a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:59.797430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 686/1372] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:43:59.803442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e554a-3864-4879-b979-63e62f7b33d2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:43:59.803749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e554a-3864-4879-b979-63e62f7b33d2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:43:59.803996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e554a-3864-4879-b979-63e62f7b33d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:43:59.823300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-713e554a-3864-4879-b979-63e62f7b33d2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:43:59.824521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-713e554a-3864-4879-b979-63e62f7b33d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:43:59.832829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 687/1373] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:00.155386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300d1e-7794-4148-8c47-9c23dfab8c09 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:00.155386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300d1e-7794-4148-8c47-9c23dfab8c09 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:00.155386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300d1e-7794-4148-8c47-9c23dfab8c09 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:00.157162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 687/1374] 10.222.0.36 () {54 vars in 963 bytes} [Mon May 23 10:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:00.162868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996305c-def3-46ab-a272-93a734c23482 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:00.163157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996305c-def3-46ab-a272-93a734c23482 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:00.163429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996305c-def3-46ab-a272-93a734c23482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:00.171076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:44:00.213493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f996305c-def3-46ab-a272-93a734c23482 None None] MFA Rules not processed for user `0a4b93133aa146bca7ee0f205cf623a8`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:00.214749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f996305c-def3-46ab-a272-93a734c23482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:00.281640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 688/1375] 10.222.0.36 () {58 vars in 1060 bytes} [Mon May 23 10:44:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:00.289425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300d1e-7794-4148-8c47-9c23dfab8c09 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:00.302147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300d1e-7794-4148-8c47-9c23dfab8c09 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:00.306306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:00.307395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:00.307958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:00.308238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:00.319195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:00.321424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a387a89-1da6-4f83-99a6-fece2ffdab69 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:00.339625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 688/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:44:01.808836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689796-9d9e-4b1c-b0eb-8bc8c68c186f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:01.809763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689796-9d9e-4b1c-b0eb-8bc8c68c186f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:01.810073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689796-9d9e-4b1c-b0eb-8bc8c68c186f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:01.811449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 689/1377] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 10:44:01.816734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b49a8-2d70-48ef-8194-81d461e6ec1f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:01.817174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b49a8-2d70-48ef-8194-81d461e6ec1f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:01.817278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b49a8-2d70-48ef-8194-81d461e6ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:01.834866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-083b49a8-2d70-48ef-8194-81d461e6ec1f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:01.836070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-083b49a8-2d70-48ef-8194-81d461e6ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:01.844020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 689/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:01.889786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee689796-9d9e-4b1c-b0eb-8bc8c68c186f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:01.905115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee689796-9d9e-4b1c-b0eb-8bc8c68c186f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:01.910721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:01.911397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:01.911679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:01.911951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:01.921638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:01.923323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ee689-8e35-43b0-a922-db22cad1e39d service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:01.934172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 690/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:04.219909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c43de-71db-44ac-bc6d-0418d297f389 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:04.220441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c43de-71db-44ac-bc6d-0418d297f389 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:04.220626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c43de-71db-44ac-bc6d-0418d297f389 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:04.222295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 690/1380] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:04.227342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd0f7-1aa9-4e9a-8a32-8e4ce7c6c795 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:04.228131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd0f7-1aa9-4e9a-8a32-8e4ce7c6c795 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:04.228469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccd0f7-1aa9-4e9a-8a32-8e4ce7c6c795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:04.255901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-dfccd0f7-1aa9-4e9a-8a32-8e4ce7c6c795 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:04.257229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dfccd0f7-1aa9-4e9a-8a32-8e4ce7c6c795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:04.266878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 691/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:04.312087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c43de-71db-44ac-bc6d-0418d297f389 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:04.327524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c43de-71db-44ac-bc6d-0418d297f389 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:04.333032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:04.333850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:04.334346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:04.334786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:04.343269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:04.345018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ead77-c273-4c69-878b-a15418667817 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:04.355133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 691/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:06.582512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9407-5618-46e9-b8a0-f986b86edd82 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:06.583022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9407-5618-46e9-b8a0-f986b86edd82 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:06.583140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d9407-5618-46e9-b8a0-f986b86edd82 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:06.584422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 692/1383] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:06.590349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f062b2-bb5b-4293-aefb-45b0a41c461a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:06.590773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f062b2-bb5b-4293-aefb-45b0a41c461a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:06.590910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f062b2-bb5b-4293-aefb-45b0a41c461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:06.613540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c9f062b2-bb5b-4293-aefb-45b0a41c461a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:06.614961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c9f062b2-bb5b-4293-aefb-45b0a41c461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:06.623449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 692/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:06.664413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d9407-5618-46e9-b8a0-f986b86edd82 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:06.680546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d9407-5618-46e9-b8a0-f986b86edd82 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:06.686237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:06.686878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:06.687162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:06.687441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:06.696151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:06.697763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5450c-bc60-41fc-b8e2-ee6476c51f3c service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:06.708061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 693/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:09.109422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a458-fb75-49bb-bc09-15f4d62a4efa None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:09.110144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a458-fb75-49bb-bc09-15f4d62a4efa None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:09.110144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0a458-fb75-49bb-bc09-15f4d62a4efa None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:09.111483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 693/1386] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:09.117517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0306-e6b8-4254-af46-7a208e2890de None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:09.117984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0306-e6b8-4254-af46-7a208e2890de None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:09.118111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c0306-e6b8-4254-af46-7a208e2890de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:09.136589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b51c0306-e6b8-4254-af46-7a208e2890de None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:09.137885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b51c0306-e6b8-4254-af46-7a208e2890de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:09.146149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 694/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:09.190367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0a458-fb75-49bb-bc09-15f4d62a4efa None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:09.208301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0a458-fb75-49bb-bc09-15f4d62a4efa None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:09.215105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:09.215911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:09.216273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:09.216631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:09.227154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:09.229836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97ae9f-25dd-4a51-a5ae-e11ad5f5e368 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:09.243166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 694/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:11.461824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd5ab-b3af-4970-a3ef-e6cbeb0d6951 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:11.462418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd5ab-b3af-4970-a3ef-e6cbeb0d6951 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:11.462418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dd5ab-b3af-4970-a3ef-e6cbeb0d6951 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:11.464332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 695/1389] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:11.470558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b639-8476-4517-b984-bf261fb945fd None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:11.471039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b639-8476-4517-b984-bf261fb945fd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:11.471563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b639-8476-4517-b984-bf261fb945fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:11.498021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-92b9b639-8476-4517-b984-bf261fb945fd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:11.499891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-92b9b639-8476-4517-b984-bf261fb945fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:11.513766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 695/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:11.557383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dd5ab-b3af-4970-a3ef-e6cbeb0d6951 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:11.571830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dd5ab-b3af-4970-a3ef-e6cbeb0d6951 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:11.578041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:11.578648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:11.578932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:11.579210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:11.588367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:11.589972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfc922-ee69-4c46-8844-8bd029de3a72 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:11.599780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 696/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:13.890207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1ad2-b932-4be9-a577-eb2dd1dc9d73 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:13.890850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1ad2-b932-4be9-a577-eb2dd1dc9d73 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:13.890850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb1ad2-b932-4be9-a577-eb2dd1dc9d73 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:13.892415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 696/1392] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:13.899495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fcfb-bf18-4cbd-9884-0cfcc99c50be None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:13.899931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fcfb-bf18-4cbd-9884-0cfcc99c50be None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:13.900718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98fcfb-bf18-4cbd-9884-0cfcc99c50be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:13.918823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2a98fcfb-bf18-4cbd-9884-0cfcc99c50be None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:13.920029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2a98fcfb-bf18-4cbd-9884-0cfcc99c50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:13.928028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 697/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:13.996995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb1ad2-b932-4be9-a577-eb2dd1dc9d73 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:14.014066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb1ad2-b932-4be9-a577-eb2dd1dc9d73 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:14.020229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:14.020870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:14.021158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:14.021435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:14.030606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:14.032251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb1003-6817-4d1f-8f2c-b73307c5baee service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:14.046393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 697/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:44:16.286706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dbde-50d8-47a2-b04b-6eb6b650dd25 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.287296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dbde-50d8-47a2-b04b-6eb6b650dd25 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.287296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293dbde-50d8-47a2-b04b-6eb6b650dd25 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.288668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 698/1395] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:16.294570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e93e1-02aa-4eec-af18-83d63d51fd87 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.295037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e93e1-02aa-4eec-af18-83d63d51fd87 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.295130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e93e1-02aa-4eec-af18-83d63d51fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.301186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=52 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:44:16.314643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-076e93e1-02aa-4eec-af18-83d63d51fd87 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:16.315863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-076e93e1-02aa-4eec-af18-83d63d51fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:16.323878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 698/1396] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:16.368266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293dbde-50d8-47a2-b04b-6eb6b650dd25 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:16.382414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293dbde-50d8-47a2-b04b-6eb6b650dd25 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:16.387808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:16.388450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.388734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.389025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.397773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:16.399447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f3d94-76e5-4f33-afb2-d725e284874f service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:16.409628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 699/1397] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:16.426220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e6f-39b7-4211-a47f-53f552151903 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.426506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e6f-39b7-4211-a47f-53f552151903 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.426807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f811e6f-39b7-4211-a47f-53f552151903 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.428177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 699/1398] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:16.433544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c53fd0-9c00-4a69-8bcd-359f85161386 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.433961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c53fd0-9c00-4a69-8bcd-359f85161386 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.434165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c53fd0-9c00-4a69-8bcd-359f85161386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.452163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c4c53fd0-9c00-4a69-8bcd-359f85161386 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:16.453364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c4c53fd0-9c00-4a69-8bcd-359f85161386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:16.462038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 700/1399] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:16.519930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d265f1-46df-4e6d-8bef-1cb8a099aea7 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.520215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d265f1-46df-4e6d-8bef-1cb8a099aea7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.520490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d265f1-46df-4e6d-8bef-1cb8a099aea7 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.521881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 700/1400] 10.222.0.36 () {54 vars in 962 bytes} [Mon May 23 10:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:16.527613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab705069-1808-4b9e-a1b6-ab1201047e10 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.527901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab705069-1808-4b9e-a1b6-ab1201047e10 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.528178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab705069-1808-4b9e-a1b6-ab1201047e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.581903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ab705069-1808-4b9e-a1b6-ab1201047e10 None None] MFA Rules not processed for user `069791ace6bc45a494dd77b732302466`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:16.583160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ab705069-1808-4b9e-a1b6-ab1201047e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:16.618083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:44:16.650685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 701/1401] 10.222.0.36 () {58 vars in 1059 bytes} [Mon May 23 10:44:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:16.658398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d265f1-46df-4e6d-8bef-1cb8a099aea7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:16.670237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d265f1-46df-4e6d-8bef-1cb8a099aea7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:16.673754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:16.674418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:16.674700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:16.674975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:16.684231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:16.685706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe27f0d-802f-4892-8d82-53607521dea6 service glance] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:16.695317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 701/1402] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:17.221651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2909db1-e98c-4800-b50d-552085629762 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:17.221941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2909db1-e98c-4800-b50d-552085629762 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:17.222247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2909db1-e98c-4800-b50d-552085629762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:17.292879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c2909db1-e98c-4800-b50d-552085629762 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:17.294430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c2909db1-e98c-4800-b50d-552085629762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:17.372130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 702/1403] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:44:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:17.413570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1936-e983-48a9-85cf-9932c0588544 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:17.414197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1936-e983-48a9-85cf-9932c0588544 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:17.414197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d1936-e983-48a9-85cf-9932c0588544 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:17.416202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 702/1404] 10.222.0.36 () {54 vars in 942 bytes} [Mon May 23 10:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:17.437212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b97a9-d983-44fb-a733-3e634a6f5642 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:17.437504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b97a9-d983-44fb-a733-3e634a6f5642 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:17.437831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b97a9-d983-44fb-a733-3e634a6f5642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:17.492291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-694b97a9-d983-44fb-a733-3e634a6f5642 None None] MFA Rules not processed for user `ef2bef098cce41279cc9a5d81f0dd782`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:17.494550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-694b97a9-d983-44fb-a733-3e634a6f5642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:17.569289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 703/1405] 10.222.0.36 () {58 vars in 1039 bytes} [Mon May 23 10:44:17 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:17.579224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d1936-e983-48a9-85cf-9932c0588544 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:17.592776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d1936-e983-48a9-85cf-9932c0588544 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:17.597627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:17.598293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:17.598576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:17.598852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:17.606874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:17.608447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ceee4-4343-4ca6-8098-678e8022ae13 service swift] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:17.616572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 703/1406] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:44:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:44:18.646902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91004-51b9-4337-bb3a-22fbec94cd0f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:18.647433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91004-51b9-4337-bb3a-22fbec94cd0f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:18.647433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91004-51b9-4337-bb3a-22fbec94cd0f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:18.652600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 704/1407] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:18.655154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54b0b-9b7a-4607-aec0-7c9eaa0f8624 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:18.655636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54b0b-9b7a-4607-aec0-7c9eaa0f8624 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:18.655744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d54b0b-9b7a-4607-aec0-7c9eaa0f8624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:18.674410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-38d54b0b-9b7a-4607-aec0-7c9eaa0f8624 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:18.675915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-38d54b0b-9b7a-4607-aec0-7c9eaa0f8624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:18.685005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 704/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:18.730364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a91004-51b9-4337-bb3a-22fbec94cd0f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:18.745421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a91004-51b9-4337-bb3a-22fbec94cd0f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:18.752321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:18.752971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:18.753261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:18.753542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:18.766692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:18.768294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aab4a4-5025-4d96-89a3-f12ee11587a9 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:18.773451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:44:18.779081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 705/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:28.669195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=17 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:44:40.711336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d597d-5fa9-4845-9b65-108974db0287 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:40.712639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d597d-5fa9-4845-9b65-108974db0287 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:40.712639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d597d-5fa9-4845-9b65-108974db0287 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:40.713972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 705/1410] 10.222.0.36 () {54 vars in 918 bytes} [Mon May 23 10:44:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:40.719096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2e43-eeee-4d03-a463-dc71e18aa0d4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:40.719584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2e43-eeee-4d03-a463-dc71e18aa0d4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:40.719693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac2e43-eeee-4d03-a463-dc71e18aa0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:40.737249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-66ac2e43-eeee-4d03-a463-dc71e18aa0d4 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:40.738960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-66ac2e43-eeee-4d03-a463-dc71e18aa0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:40.747388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 706/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 23 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:40.761776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567d07-eb72-48c6-80ad-6c5f3b8cb86f None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:40.762099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567d07-eb72-48c6-80ad-6c5f3b8cb86f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:40.762383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47567d07-eb72-48c6-80ad-6c5f3b8cb86f None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:40.763734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 706/1412] 10.222.0.36 () {54 vars in 918 bytes} [Mon May 23 10:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:40.768515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871700c-332a-4603-9336-fde7a02bb1e4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:40.768805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871700c-332a-4603-9336-fde7a02bb1e4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:40.769084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871700c-332a-4603-9336-fde7a02bb1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:40.786941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1871700c-332a-4603-9336-fde7a02bb1e4 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:40.788113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1871700c-332a-4603-9336-fde7a02bb1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:40.797089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 707/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Mon May 23 10:44:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:43.441685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b98d-0bf6-4cf2-87a4-cccf21cad9c0 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:43.442242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b98d-0bf6-4cf2-87a4-cccf21cad9c0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:43.442365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9b98d-0bf6-4cf2-87a4-cccf21cad9c0 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:43.443680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:43.449562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a8c0-a207-42b2-9a6a-ce84af28fd12 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:43.450020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a8c0-a207-42b2-9a6a-ce84af28fd12 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:43.450146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00a8c0-a207-42b2-9a6a-ce84af28fd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:43.467826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2a00a8c0-a207-42b2-9a6a-ce84af28fd12 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:43.469056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2a00a8c0-a207-42b2-9a6a-ce84af28fd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:43.476822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:43.519632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b98d-0bf6-4cf2-87a4-cccf21cad9c0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:43.534063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9b98d-0bf6-4cf2-87a4-cccf21cad9c0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:43.539480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:43.540645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:43.540912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:43.541189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:43.549621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:43.551323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ddee89-35ac-4cd6-bad2-d413195ab57f service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:43.561714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 708/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:44.521417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba869-8960-4937-8398-ee0707df6f4a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:44.521981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba869-8960-4937-8398-ee0707df6f4a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:44.521981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ba869-8960-4937-8398-ee0707df6f4a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:44.523310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:44.528176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9451-250e-4a25-a8a6-5b8985a68135 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:44.528572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9451-250e-4a25-a8a6-5b8985a68135 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:44.528738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c9451-250e-4a25-a8a6-5b8985a68135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:44.548180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-288c9451-250e-4a25-a8a6-5b8985a68135 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:44.549385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-288c9451-250e-4a25-a8a6-5b8985a68135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:44.557839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:44.733006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ba869-8960-4937-8398-ee0707df6f4a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:44.747070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ba869-8960-4937-8398-ee0707df6f4a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:44.752337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:44.752963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:44.753245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:44.753521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:44.761398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:44.763060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887af53d-2706-4cdc-a476-c650d7144720 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:44.772476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 710/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:45.693440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530cd7-f01c-494c-95f7-6dbd1ec5201a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:45.694034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530cd7-f01c-494c-95f7-6dbd1ec5201a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:45.694034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a530cd7-f01c-494c-95f7-6dbd1ec5201a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:45.695412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:45.700915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af3fd-542f-4405-88a8-8a4b262935b2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:45.701337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af3fd-542f-4405-88a8-8a4b262935b2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:45.701494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1af3fd-542f-4405-88a8-8a4b262935b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:45.719169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9d1af3fd-542f-4405-88a8-8a4b262935b2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:45.720368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9d1af3fd-542f-4405-88a8-8a4b262935b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:45.728162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:45.877458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a530cd7-f01c-494c-95f7-6dbd1ec5201a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:45.892106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a530cd7-f01c-494c-95f7-6dbd1ec5201a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:45.897420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:45.898080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:45.898366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:45.898644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:45.906616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:45.908207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c938a05-1d8a-42ec-b89c-32741e45dded service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:45.917920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 711/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:44:46.812378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287161-c7e4-428a-b990-1b752ecf77cb None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:46.812987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287161-c7e4-428a-b990-1b752ecf77cb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:46.812987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a287161-c7e4-428a-b990-1b752ecf77cb None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:46.814374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:46.819415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49467dd9-57e7-419d-8566-2cd562840adc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:46.819849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49467dd9-57e7-419d-8566-2cd562840adc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:46.819963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49467dd9-57e7-419d-8566-2cd562840adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:46.839628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-49467dd9-57e7-419d-8566-2cd562840adc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:46.840822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-49467dd9-57e7-419d-8566-2cd562840adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:46.848594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:46.997205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a287161-c7e4-428a-b990-1b752ecf77cb None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:47.011959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a287161-c7e4-428a-b990-1b752ecf77cb None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:47.017541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:47.018199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:47.018482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:47.018757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:47.026692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:47.028264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca49885-458c-4b05-a474-8efff2332f60 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:47.037306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 713/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:47.931500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f5a3-cd83-42d4-bbb1-1be57e8ebaae None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:47.932073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f5a3-cd83-42d4-bbb1-1be57e8ebaae None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:47.932073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69f5a3-cd83-42d4-bbb1-1be57e8ebaae None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:47.933419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:47.938374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58d6-0062-44a5-bc3a-c4931f784c03 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:47.938843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58d6-0062-44a5-bc3a-c4931f784c03 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:47.938951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e58d6-0062-44a5-bc3a-c4931f784c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:47.957331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-db9e58d6-0062-44a5-bc3a-c4931f784c03 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:47.958567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-db9e58d6-0062-44a5-bc3a-c4931f784c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:47.966445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:48.112556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f5a3-cd83-42d4-bbb1-1be57e8ebaae None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:48.127218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69f5a3-cd83-42d4-bbb1-1be57e8ebaae None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:48.132592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:48.133225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:48.133512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:48.133787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:48.142087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:48.143673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d2078-d075-4412-af56-488f4777c6c3 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:48.152898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 714/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:48.994871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ebd-e19e-4ce9-bf15-709333ead6de None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:48.995417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ebd-e19e-4ce9-bf15-709333ead6de None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:48.995417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea4ebd-e19e-4ce9-bf15-709333ead6de None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:48.996790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:49.001607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c767c-3e63-432a-b784-cb2701171f5a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:49.002082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c767c-3e63-432a-b784-cb2701171f5a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:49.002219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c767c-3e63-432a-b784-cb2701171f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:49.020485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7c0c767c-3e63-432a-b784-cb2701171f5a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:49.021683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7c0c767c-3e63-432a-b784-cb2701171f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:49.029516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:49.177757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4ebd-e19e-4ce9-bf15-709333ead6de None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:49.191889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea4ebd-e19e-4ce9-bf15-709333ead6de None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:49.197618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:49.198278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:49.198563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:49.198836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:49.206784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:49.208449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d67dd-5e56-4105-8e7c-3202eb18ba5f service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:49.218189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 716/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:50.060051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8e39-fdc1-45cd-88d4-6ed260372ef5 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:50.060622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8e39-fdc1-45cd-88d4-6ed260372ef5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:50.060622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb8e39-fdc1-45cd-88d4-6ed260372ef5 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:50.062007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:50.066997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ace8d-31ac-484a-8b24-8e69ddbf2b9b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:50.067423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ace8d-31ac-484a-8b24-8e69ddbf2b9b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:50.067563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ace8d-31ac-484a-8b24-8e69ddbf2b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:50.085680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b42ace8d-31ac-484a-8b24-8e69ddbf2b9b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:50.086903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b42ace8d-31ac-484a-8b24-8e69ddbf2b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:50.094620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:50.243976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8e39-fdc1-45cd-88d4-6ed260372ef5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:50.257888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb8e39-fdc1-45cd-88d4-6ed260372ef5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:50.263839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:50.264465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:50.264747 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:50.265021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:50.273177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:50.274783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9da9c-e0a2-4817-9e0a-e5e182f7dcad service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:50.283930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 717/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:51.142835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405618d-c8d2-4bf6-ae22-258cac704cd8 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:51.143408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405618d-c8d2-4bf6-ae22-258cac704cd8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:51.143408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405618d-c8d2-4bf6-ae22-258cac704cd8 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:51.144813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:51.149702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360540a1-b023-465e-ac6c-1943eb524805 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:51.150129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360540a1-b023-465e-ac6c-1943eb524805 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:51.150293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360540a1-b023-465e-ac6c-1943eb524805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:51.168357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-360540a1-b023-465e-ac6c-1943eb524805 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:51.169542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-360540a1-b023-465e-ac6c-1943eb524805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:51.177240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:51.324380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405618d-c8d2-4bf6-ae22-258cac704cd8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:51.338392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405618d-c8d2-4bf6-ae22-258cac704cd8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:51.343624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:51.344250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:51.344532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:51.344808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:51.353146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:51.354766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3c61f-716a-48df-b683-5cc293031aa0 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:51.363920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 719/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:52.198181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abf54-3fbf-439c-ad40-b82f2ff01bb2 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:52.198716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abf54-3fbf-439c-ad40-b82f2ff01bb2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:52.198716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36abf54-3fbf-439c-ad40-b82f2ff01bb2 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:52.200163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:52.204997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34af5-2b82-47d4-b094-b51fdc661725 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:52.205412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34af5-2b82-47d4-b094-b51fdc661725 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:52.205547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34af5-2b82-47d4-b094-b51fdc661725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:52.223131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d5b34af5-2b82-47d4-b094-b51fdc661725 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:52.224312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d5b34af5-2b82-47d4-b094-b51fdc661725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:52.234001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:52.378629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abf54-3fbf-439c-ad40-b82f2ff01bb2 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:52.393651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36abf54-3fbf-439c-ad40-b82f2ff01bb2 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:52.399081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:52.399713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:52.399994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:52.400270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:52.408236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:52.409859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db68f0-9f10-42e8-b7f7-d2bbff3987b6 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:52.419154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 720/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:54.472850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d41e-e417-4e6a-86dd-d3a98badccfc None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:54.473381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d41e-e417-4e6a-86dd-d3a98badccfc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:54.473381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d41e-e417-4e6a-86dd-d3a98badccfc None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:54.474805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:54.480197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde53ed-f248-499c-9568-251d35f66893 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:54.480627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde53ed-f248-499c-9568-251d35f66893 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:54.480794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde53ed-f248-499c-9568-251d35f66893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:54.498967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ffde53ed-f248-499c-9568-251d35f66893 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:54.500106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ffde53ed-f248-499c-9568-251d35f66893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:54.507941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:54.545815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d41e-e417-4e6a-86dd-d3a98badccfc None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:54.556879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d41e-e417-4e6a-86dd-d3a98badccfc None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:54.560358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:54.560981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:54.561260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:54.561533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:54.570179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:54.572257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ca4dd-88cd-4707-88bb-2c4dbb03e18d service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:54.581340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 722/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:56.941140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1036a5-22e2-441e-a5c1-ee18d85f7a2a None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:56.941655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1036a5-22e2-441e-a5c1-ee18d85f7a2a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:56.941655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1036a5-22e2-441e-a5c1-ee18d85f7a2a None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:56.943050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:56.948397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6dc6-80c6-49bc-94c1-51485ffe3d14 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:56.948828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6dc6-80c6-49bc-94c1-51485ffe3d14 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:56.948934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6dc6-80c6-49bc-94c1-51485ffe3d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:56.966593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-739c6dc6-80c6-49bc-94c1-51485ffe3d14 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:56.967766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-739c6dc6-80c6-49bc-94c1-51485ffe3d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:56.975400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:57.012173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1036a5-22e2-441e-a5c1-ee18d85f7a2a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:57.023296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1036a5-22e2-441e-a5c1-ee18d85f7a2a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:57.027421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:57.028030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:57.028334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:57.028590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:57.037370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:57.038849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99f7b3-9f9e-4d1e-809f-be0a615db6d9 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:57.048711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 723/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:44:59.630973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceab29-cc2a-41e4-89e3-315a5c78a1eb None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:59.631545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceab29-cc2a-41e4-89e3-315a5c78a1eb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:59.631545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceab29-cc2a-41e4-89e3-315a5c78a1eb None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:59.632917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:44:59.638372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272fd2-48b1-49dd-8488-ef0ae134ef3e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:59.638761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272fd2-48b1-49dd-8488-ef0ae134ef3e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:59.638926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272fd2-48b1-49dd-8488-ef0ae134ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:59.656830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-93272fd2-48b1-49dd-8488-ef0ae134ef3e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:44:59.658039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-93272fd2-48b1-49dd-8488-ef0ae134ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:44:59.666341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:44:59.703984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceab29-cc2a-41e4-89e3-315a5c78a1eb None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:44:59.715263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceab29-cc2a-41e4-89e3-315a5c78a1eb None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:44:59.718798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:44:59.720440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:44:59.720728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:44:59.721005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:44:59.730673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:44:59.732108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb97a5-aed7-4f46-9e84-d86716c55a27 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:44:59.741713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 725/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:02.046160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b89ed-aa87-47bf-b3c4-88663f26be0b None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:02.046778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b89ed-aa87-47bf-b3c4-88663f26be0b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:02.046778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b89ed-aa87-47bf-b3c4-88663f26be0b None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:02.048107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:02.053273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ea0c-6f32-46b8-841a-6be65f440db5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:02.053694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ea0c-6f32-46b8-841a-6be65f440db5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:02.053839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929ea0c-6f32-46b8-841a-6be65f440db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:02.072501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5929ea0c-6f32-46b8-841a-6be65f440db5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:02.073681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5929ea0c-6f32-46b8-841a-6be65f440db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:02.081385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:02.119032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b89ed-aa87-47bf-b3c4-88663f26be0b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:02.129836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b89ed-aa87-47bf-b3c4-88663f26be0b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:02.133971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:02.134610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:02.134923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:02.135201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:02.144418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:02.145844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addea83b-2874-4274-8730-f088455bc016 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:02.155114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 726/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:04.383414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491cbc-6598-4134-9efb-ce6229f1ea2f None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:04.383919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491cbc-6598-4134-9efb-ce6229f1ea2f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:04.384028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491cbc-6598-4134-9efb-ce6229f1ea2f None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:04.385346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:04.390649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763168-5f40-4300-9065-a9f4c6332c33 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:04.391050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763168-5f40-4300-9065-a9f4c6332c33 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:04.391210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be763168-5f40-4300-9065-a9f4c6332c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:04.409124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-be763168-5f40-4300-9065-a9f4c6332c33 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:04.410347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-be763168-5f40-4300-9065-a9f4c6332c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:04.418656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:04.454835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491cbc-6598-4134-9efb-ce6229f1ea2f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:04.466172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491cbc-6598-4134-9efb-ce6229f1ea2f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:04.469929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:04.470587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:04.470874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:04.471159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:04.480477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:04.481898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd53635-4c3a-4950-9c02-2c1a2e8b1b20 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:04.491162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 728/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:06.814192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea978ae-951b-44ad-bf59-18eecb6263ac None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:06.814757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea978ae-951b-44ad-bf59-18eecb6263ac None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:06.814757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea978ae-951b-44ad-bf59-18eecb6263ac None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:06.816163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:06.821885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aac21e-ca59-4a13-98c7-ab7b06b33f77 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:06.822357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aac21e-ca59-4a13-98c7-ab7b06b33f77 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:06.822465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aac21e-ca59-4a13-98c7-ab7b06b33f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:06.840296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c0aac21e-ca59-4a13-98c7-ab7b06b33f77 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:06.841476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c0aac21e-ca59-4a13-98c7-ab7b06b33f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:06.849462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:06.886391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea978ae-951b-44ad-bf59-18eecb6263ac None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:06.897131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea978ae-951b-44ad-bf59-18eecb6263ac None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:06.901166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:06.901793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:06.902118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:06.902400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:06.911151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:06.912585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15f253-9fb7-44ce-aba4-2ea60e1df2b3 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:06.922935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 729/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:09.145459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0898-ecf4-4780-93f4-2a39d4f9a641 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:09.146087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0898-ecf4-4780-93f4-2a39d4f9a641 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:09.146087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0898-ecf4-4780-93f4-2a39d4f9a641 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:09.147412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:09.153180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec15c7c-bf2c-43f5-bc63-ec7ca20d167c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:09.153619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec15c7c-bf2c-43f5-bc63-ec7ca20d167c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:09.153750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec15c7c-bf2c-43f5-bc63-ec7ca20d167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:09.159879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=70 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:45:09.172498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0ec15c7c-bf2c-43f5-bc63-ec7ca20d167c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:09.173695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0ec15c7c-bf2c-43f5-bc63-ec7ca20d167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:09.181513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:09.218528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e0898-ecf4-4780-93f4-2a39d4f9a641 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:09.223857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=73 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:45:09.230282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e0898-ecf4-4780-93f4-2a39d4f9a641 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:09.233977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:09.234588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:09.234885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:09.235163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:09.244445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:09.245873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351881-a2c5-4bab-b5b1-c51633d911d8 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:09.255473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 731/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:11.487638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3019bfa-0deb-4410-88e2-e02ee7f3564e None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:11.488190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3019bfa-0deb-4410-88e2-e02ee7f3564e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:11.488190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3019bfa-0deb-4410-88e2-e02ee7f3564e None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:11.489572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:11.495597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f388de-6c81-475c-94c9-50528d631fed None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:11.496032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f388de-6c81-475c-94c9-50528d631fed None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:11.496130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f388de-6c81-475c-94c9-50528d631fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:11.513687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-00f388de-6c81-475c-94c9-50528d631fed None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:11.514915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-00f388de-6c81-475c-94c9-50528d631fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:11.523022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:11.559710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3019bfa-0deb-4410-88e2-e02ee7f3564e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:11.570163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3019bfa-0deb-4410-88e2-e02ee7f3564e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:11.573635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:11.574295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:11.574576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:11.574848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:11.583420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:11.584840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c6995-f1ed-4b32-93dc-5f2803f82488 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:11.594005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 732/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:13.792251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f371-08df-4679-b5d2-3a8812d634bb None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:13.792821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f371-08df-4679-b5d2-3a8812d634bb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:13.792821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f371-08df-4679-b5d2-3a8812d634bb None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:13.794251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:13.799408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c15d0d-77cf-4244-b929-7f51f29a7c50 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:13.799794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c15d0d-77cf-4244-b929-7f51f29a7c50 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:13.799958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c15d0d-77cf-4244-b929-7f51f29a7c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:13.817852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-29c15d0d-77cf-4244-b929-7f51f29a7c50 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:13.819053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-29c15d0d-77cf-4244-b929-7f51f29a7c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:13.827942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:13.863361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451f371-08df-4679-b5d2-3a8812d634bb None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:13.873871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451f371-08df-4679-b5d2-3a8812d634bb None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:13.877360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:13.878020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:13.878309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:13.878587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:13.887777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:13.889197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104efc6-dc65-4183-bf02-740687f60905 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:13.899369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 734/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:16.092230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3dbe0-f4dd-4a8e-a98e-7b23079a76b1 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:16.092810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3dbe0-f4dd-4a8e-a98e-7b23079a76b1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:16.092810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3dbe0-f4dd-4a8e-a98e-7b23079a76b1 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:16.094205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:16.099581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47ebb-803e-43c1-8eef-b5510dd60470 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:16.099984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47ebb-803e-43c1-8eef-b5510dd60470 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:16.100145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47ebb-803e-43c1-8eef-b5510dd60470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:16.119258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9fd47ebb-803e-43c1-8eef-b5510dd60470 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:16.120441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9fd47ebb-803e-43c1-8eef-b5510dd60470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:16.128155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:16.165465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3dbe0-f4dd-4a8e-a98e-7b23079a76b1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:16.176084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3dbe0-f4dd-4a8e-a98e-7b23079a76b1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:16.179567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:16.180502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:16.180785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:16.181055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:16.189671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:16.191129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8b577-07ea-4905-af50-7cfa5773cbf9 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:16.200398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 735/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:26.112996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=17 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:45:26.196279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=17 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:45:48.802933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7324e5-7535-4d1f-8d44-aca248450dd6 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:48.803532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7324e5-7535-4d1f-8d44-aca248450dd6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:48.803532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7324e5-7535-4d1f-8d44-aca248450dd6 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:48.804871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 736/1471] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:48.810586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760d3c-1f4c-424e-afa7-6232f9ae5591 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:48.811029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760d3c-1f4c-424e-afa7-6232f9ae5591 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:48.811142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760d3c-1f4c-424e-afa7-6232f9ae5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:48.828463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ce760d3c-1f4c-424e-afa7-6232f9ae5591 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:48.829655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce760d3c-1f4c-424e-afa7-6232f9ae5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:48.837477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 736/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:48.883439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be155-260a-49b4-9d5d-27cd7ea8ebc0 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:48.883722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be155-260a-49b4-9d5d-27cd7ea8ebc0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:48.883996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6be155-260a-49b4-9d5d-27cd7ea8ebc0 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:48.885342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 737/1473] 10.222.0.36 () {54 vars in 962 bytes} [Mon May 23 10:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:48.890143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13670-6aa6-4d32-b5d9-7e9cf5dc2593 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:48.890430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13670-6aa6-4d32-b5d9-7e9cf5dc2593 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:48.890705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca13670-6aa6-4d32-b5d9-7e9cf5dc2593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:48.908552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bca13670-6aa6-4d32-b5d9-7e9cf5dc2593 None None] MFA Rules not processed for user `069791ace6bc45a494dd77b732302466`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:48.909717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bca13670-6aa6-4d32-b5d9-7e9cf5dc2593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:48.916570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 737/1474] 10.222.0.36 () {58 vars in 1059 bytes} [Mon May 23 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:48.923672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be155-260a-49b4-9d5d-27cd7ea8ebc0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:48.934967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6be155-260a-49b4-9d5d-27cd7ea8ebc0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:48.938443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:48.939057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:48.939337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:48.939606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:48.948156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:48.949574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9db82-ef9d-41eb-ac85-5364047b1444 service glance] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:48.958633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 738/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:45:51.216145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add110c-0fef-46b7-8dc7-b04ce745be77 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:51.216713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add110c-0fef-46b7-8dc7-b04ce745be77 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:51.216713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add110c-0fef-46b7-8dc7-b04ce745be77 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:51.218105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 738/1476] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:51.223326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9d1-c149-4fe9-8b20-1d4f33a6444e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:51.223776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9d1-c149-4fe9-8b20-1d4f33a6444e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:51.223916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466dc9d1-c149-4fe9-8b20-1d4f33a6444e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:51.241583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-466dc9d1-c149-4fe9-8b20-1d4f33a6444e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:51.242841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-466dc9d1-c149-4fe9-8b20-1d4f33a6444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:51.250617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 739/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:51.273658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add110c-0fef-46b7-8dc7-b04ce745be77 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:51.286627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add110c-0fef-46b7-8dc7-b04ce745be77 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:51.291082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:51.291731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:51.292017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:51.292294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:51.293026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:51.294826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:45:51.294826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:45:51.294826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:45:51.295977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd33be-14ff-413f-9799-e2fd61804c94 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:51.314504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 739/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Mon May 23 10:45:51 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:45:53.512206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7828220-2d74-4de9-9553-1cde9e2e305a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:53.512760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7828220-2d74-4de9-9553-1cde9e2e305a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:53.512760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7828220-2d74-4de9-9553-1cde9e2e305a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:53.514077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 740/1479] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:53.519762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e229-0468-4b30-a736-88e0349f29fd None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:53.520183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e229-0468-4b30-a736-88e0349f29fd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:53.520324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e229-0468-4b30-a736-88e0349f29fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:53.539263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6f45e229-0468-4b30-a736-88e0349f29fd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:53.540457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6f45e229-0468-4b30-a736-88e0349f29fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:53.548167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 740/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:53.591303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7828220-2d74-4de9-9553-1cde9e2e305a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:53.605503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7828220-2d74-4de9-9553-1cde9e2e305a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:53.611202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:53.611841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:53.612126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:53.612402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:53.621104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:53.622735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eacc85-7ec2-4fdd-8e8b-750a686da70e service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:53.631946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 741/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:55.996446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694575bc-ef51-470d-85f4-dc5dbc5a1a53 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:55.997104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694575bc-ef51-470d-85f4-dc5dbc5a1a53 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:55.997104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694575bc-ef51-470d-85f4-dc5dbc5a1a53 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:55.998528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 741/1482] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:56.003797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ea58-080e-42a7-8cf1-eef87e718ac4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:56.004239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ea58-080e-42a7-8cf1-eef87e718ac4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:56.004351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664ea58-080e-42a7-8cf1-eef87e718ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:56.022868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1664ea58-080e-42a7-8cf1-eef87e718ac4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:56.024572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1664ea58-080e-42a7-8cf1-eef87e718ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:56.032330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 742/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:56.067942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694575bc-ef51-470d-85f4-dc5dbc5a1a53 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:56.078506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694575bc-ef51-470d-85f4-dc5dbc5a1a53 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:56.082015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:56.082659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:56.082943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:56.083225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:56.093376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:56.094440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cdeba-cfe3-4b1d-8dad-9c79831dc4cd service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:56.104666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 742/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:45:58.314566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e6c2-4e6f-473b-ad62-bf7b107bfad9 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:58.315118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e6c2-4e6f-473b-ad62-bf7b107bfad9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:58.315118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e6c2-4e6f-473b-ad62-bf7b107bfad9 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:58.316477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 743/1485] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:45:58.321695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9f25-bc4d-436f-a1d2-ecf4761b2f0a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:58.322132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9f25-bc4d-436f-a1d2-ecf4761b2f0a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:58.322298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b9f25-bc4d-436f-a1d2-ecf4761b2f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:58.340279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4f7b9f25-bc4d-436f-a1d2-ecf4761b2f0a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:45:58.341390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4f7b9f25-bc4d-436f-a1d2-ecf4761b2f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:45:58.349113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 743/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:45:58.386059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3e6c2-4e6f-473b-ad62-bf7b107bfad9 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:45:58.396761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3e6c2-4e6f-473b-ad62-bf7b107bfad9 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:45:58.400780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:45:58.401409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:45:58.401688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:45:58.401970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:45:58.410623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:45:58.412042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ebdad-fa3d-4c3f-bcb1-46c1e5d5ef28 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:45:58.421972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 744/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:46:00.771766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afd074-d501-4fe9-8118-23c07305f1fb None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:00.772327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afd074-d501-4fe9-8118-23c07305f1fb None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:00.772327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afd074-d501-4fe9-8118-23c07305f1fb None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:00.773841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 744/1488] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:46:00.779272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada55cb-4547-4ece-af48-26f01c848843 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:00.779691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada55cb-4547-4ece-af48-26f01c848843 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:00.779807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada55cb-4547-4ece-af48-26f01c848843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:00.797522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1ada55cb-4547-4ece-af48-26f01c848843 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:46:00.798742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1ada55cb-4547-4ece-af48-26f01c848843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:46:00.807022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 745/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:46:00.846636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afd074-d501-4fe9-8118-23c07305f1fb None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:46:00.857223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afd074-d501-4fe9-8118-23c07305f1fb None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:46:00.860767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:46:00.861396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:00.861682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:00.861977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:00.870989 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:46:00.872423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8facaa2-d01c-4b86-a9bc-80cff31174c5 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:46:00.881856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 745/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:46:03.232079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c263-f023-4c48-974c-8f04b932d7bc None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:03.232691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c263-f023-4c48-974c-8f04b932d7bc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:03.232691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c263-f023-4c48-974c-8f04b932d7bc None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:03.234447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 746/1491] 10.222.0.36 () {54 vars in 926 bytes} [Mon May 23 10:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:46:03.240486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16d59-6d55-42f8-8e52-ba651a3caf60 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:03.241040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16d59-6d55-42f8-8e52-ba651a3caf60 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:03.241040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f16d59-6d55-42f8-8e52-ba651a3caf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:03.259757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-16f16d59-6d55-42f8-8e52-ba651a3caf60 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:46:03.260931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-16f16d59-6d55-42f8-8e52-ba651a3caf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:46:03.269210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 746/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Mon May 23 10:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:46:03.307799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5c263-f023-4c48-974c-8f04b932d7bc None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:46:03.318278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5c263-f023-4c48-974c-8f04b932d7bc None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:46:03.322264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:46:03.322892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:46:03.323171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:46:03.323442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:46:03.332046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:46:03.333462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381e5f5-67f0-4f7c-aebe-b0ddff19234a service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:46:03.343301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 747/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:46:13.254122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=33 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:46:13.339190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=33 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:47:48.511383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.511926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.511926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.529982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:47:48.531191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:48.539003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 747/1494] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:47:48.548028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.548511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.548613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.566574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:47:48.567759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:48.576196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 748/1495] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:47:48.580598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.590101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973a675-397e-4978-9883-8fbe4cef54db None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.594498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.595136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.595419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.595696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.596410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:47:48.596713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.597729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.597729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:47:48.597729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:47:48.598667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.610078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 748/1496] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:47:48.618351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.627400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039cf82-e842-484c-bb73-733b1a09b95f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.632487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.633133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.633421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.633700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.634432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.636195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.636195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:47:48.636195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:47:48.637412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.672319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 749/1497] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:47:48 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:47:48.676535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.686726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31390b-0054-41b4-8b85-6f8271ead36c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.742567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.743233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.743520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.743801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.744490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.745499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.745499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:47:48.745499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:47:48.746741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.766762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 749/1498] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:47:48 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:47:48.771181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.781192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b5994-9486-4023-b350-e766f4ed2af2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.785669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.786356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.786643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.786919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.788178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.789184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.789184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:47:48.789184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:47:48.790090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.802545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 750/1499] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:47:48.806925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.816593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd9aa0-98ce-49cd-bcbe-f8b6adb86fc4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.821530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.822195 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.822480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.822753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.823462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.824465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.824465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:47:48.824465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:47:48.825333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005d41ef-7433-4ffb-830d-b1fbfc3e874f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.838855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 750/1500] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:47:48.842775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:48.852384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f293d7d-0f45-4136-abc2-fc459527ccdb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:48.856820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:48.857688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.858002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.858283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] PATH_INFO: `/v3/projects/7a275363822e461f9e00d5f0c50be823/users/f23723559093485f89080ee88ebf1cc6/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.872620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a275363822e461f9e00d5f0c50be823, user_id=f23723559093485f89080ee88ebf1cc6, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:48.873642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:48.873642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:47:48.873642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:47:48.875634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb76bf0-e040-4b89-9db5-4593717aab36 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:48.888558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 751/1501] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:47:48 2022] PUT /identity/v3/projects/7a275363822e461f9e00d5f0c50be823/users/f23723559093485f89080ee88ebf1cc6/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:47:48.900775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195472-3588-4392-8357-71fdc4ab5adf None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:48.901073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195472-3588-4392-8357-71fdc4ab5adf None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:48.901357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195472-3588-4392-8357-71fdc4ab5adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:48.926620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-78195472-3588-4392-8357-71fdc4ab5adf None None] MFA Rules not processed for user `f23723559093485f89080ee88ebf1cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:47:48.927857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-78195472-3588-4392-8357-71fdc4ab5adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:48.990870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 751/1502] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:47:49.004030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa24c-db2b-46f1-bd47-adca33181b0a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.004348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa24c-db2b-46f1-bd47-adca33181b0a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.004656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa24c-db2b-46f1-bd47-adca33181b0a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.006552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 752/1503] 10.222.0.36 () {54 vars in 963 bytes} [Mon May 23 10:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:47:49.011191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522a66-3e32-4d11-9e15-a7d33c746a95 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.011546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522a66-3e32-4d11-9e15-a7d33c746a95 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.011770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95522a66-3e32-4d11-9e15-a7d33c746a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.031807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-95522a66-3e32-4d11-9e15-a7d33c746a95 None None] MFA Rules not processed for user `0a4b93133aa146bca7ee0f205cf623a8`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:47:49.032937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-95522a66-3e32-4d11-9e15-a7d33c746a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:49.076363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:47:49.085402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 752/1504] 10.222.0.36 () {58 vars in 1060 bytes} [Mon May 23 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:47:49.092960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaa24c-db2b-46f1-bd47-adca33181b0a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:49.103340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaa24c-db2b-46f1-bd47-adca33181b0a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:49.106813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:49.107440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.107724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.108001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.116800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:49.118259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:49.127038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 753/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:47:49.167966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d164-0bcd-493e-ba69-dbebe0c4934b None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.168261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d164-0bcd-493e-ba69-dbebe0c4934b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.168545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9d164-0bcd-493e-ba69-dbebe0c4934b None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.169842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 753/1506] 10.222.0.36 () {52 vars in 829 bytes} [Mon May 23 10:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:47:49.173369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:49.234616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aff329-8b67-41ee-9d5b-15ac7769c7e9 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:49.240130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:49.241513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.241795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.242109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] PATH_INFO: `/v3/users/f23723559093485f89080ee88ebf1cc6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.244016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23723559093485f89080ee88ebf1cc6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:49.244645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:49.244645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:47:49.244645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:47:49.245890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:49.309863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23723559093485f89080ee88ebf1cc6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.322620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23723559093485f89080ee88ebf1cc6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.323391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23723559093485f89080ee88ebf1cc6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.324012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23723559093485f89080ee88ebf1cc6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.332582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23723559093485f89080ee88ebf1cc6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.348342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:47:49.357751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 754/1507] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:47:49 2022] DELETE /identity/v3/users/f23723559093485f89080ee88ebf1cc6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:47:49.362239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.362536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.362823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.380505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:47:49.382380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:49.439449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 754/1508] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:47:49.449518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:49.450975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:49.508891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ba53a-4644-425b-9c65-2c6d73476b4d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:49.512442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:49.513075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.513359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.513634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.523369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:49.524837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effa32e-09c5-40af-9f69-d11b7f7bd69b service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:49.534111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 755/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:47:49.885841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:47:49.887804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:47:49.901479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e1854-7461-47c1-adce-3e9d12cf5c36 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:47:49.905992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:47:49.906854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:47:49.907134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:47:49.907409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] PATH_INFO: `/v3/projects/7a275363822e461f9e00d5f0c50be823` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:47:49.908583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a275363822e461f9e00d5f0c50be823)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:47:49.909607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:47:49.909607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:47:49.909607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:47:49.910855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:47:49.973177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a275363822e461f9e00d5f0c50be823', 'request_id': 'req-9b78f76c-ac59-43ff-ac77-17a0efe083da'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.987135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a275363822e461f9e00d5f0c50be823', 'request_id': 'req-9b78f76c-ac59-43ff-ac77-17a0efe083da'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:49.992978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a275363822e461f9e00d5f0c50be823', 'request_id': 'req-9b78f76c-ac59-43ff-ac77-17a0efe083da'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:50.007846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] The token cache is being invalidate because project 7a275363822e461f9e00d5f0c50be823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:47:50.008130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:50.008778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-9b78f76c-ac59-43ff-ac77-17a0efe083da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:47:50.010415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 755/1510] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:47:49 2022] DELETE /identity/v3/projects/7a275363822e461f9e00d5f0c50be823 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:47:59.530093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:48:00.005969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:49:34.064983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:34.065591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:34.065591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:34.066958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:34.072241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724024d-9574-4c2e-83e3-31b4c333ce7d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:34.072696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724024d-9574-4c2e-83e3-31b4c333ce7d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:34.072809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724024d-9574-4c2e-83e3-31b4c333ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:34.090778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9724024d-9574-4c2e-83e3-31b4c333ce7d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:34.092006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9724024d-9574-4c2e-83e3-31b4c333ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:34.149684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:34.318944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:34.320574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:34.393713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc18ba-271f-452d-84d3-de831c00589d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:34.399384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:34.400024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:34.400313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:34.400594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:34.408683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:34.410330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2109-e52b-4d00-9713-fc26416ad7d3 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:34.420308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 757/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:35.052627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5620bc-629a-4d60-be63-b112b06682d8 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:35.053065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5620bc-629a-4d60-be63-b112b06682d8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:35.053357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5620bc-629a-4d60-be63-b112b06682d8 None None] PATH_INFO: `/v3` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:35.054729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 757/1514] 10.222.0.73 () {54 vars in 925 bytes} [Mon May 23 10:49:35 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:49:35.060817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6b21-f93f-4ce9-9e60-97857f88f363 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:35.061110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6b21-f93f-4ce9-9e60-97857f88f363 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:35.061393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a6b21-f93f-4ce9-9e60-97857f88f363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:35.079580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2c5a6b21-f93f-4ce9-9e60-97857f88f363 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:35.080779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2c5a6b21-f93f-4ce9-9e60-97857f88f363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:35.144382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 758/1515] 10.222.0.73 () {58 vars in 1015 bytes} [Mon May 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:49:35.295250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8560-9a5e-4dbf-b74b-69519adaaecd None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:35.295792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8560-9a5e-4dbf-b74b-69519adaaecd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:35.295895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8560-9a5e-4dbf-b74b-69519adaaecd None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:35.297235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 758/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:35.302242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382bd24-beb5-44a4-8e9f-0d5daebcb6f5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:35.302538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382bd24-beb5-44a4-8e9f-0d5daebcb6f5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:35.302868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382bd24-beb5-44a4-8e9f-0d5daebcb6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:35.321156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5382bd24-beb5-44a4-8e9f-0d5daebcb6f5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:35.322399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5382bd24-beb5-44a4-8e9f-0d5daebcb6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:35.330347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 759/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:49:35.480469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8560-9a5e-4dbf-b74b-69519adaaecd None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:35.494781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8560-9a5e-4dbf-b74b-69519adaaecd None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:35.500191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:35.500823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:35.501110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:35.501385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:35.509825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:35.511475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95b97e-1810-4258-b713-3793baa3e837 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:35.520979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 759/1518] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:44.358458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454b3-019b-45c3-9424-a0dfee221ab5 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:44.359024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454b3-019b-45c3-9424-a0dfee221ab5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:44.359024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454b3-019b-45c3-9424-a0dfee221ab5 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:44.360401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 760/1519] 10.222.0.36 () {54 vars in 960 bytes} [Mon May 23 10:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:44.365572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b0d3-34c5-4e3b-b1af-1c3169ac8502 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:44.366041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b0d3-34c5-4e3b-b1af-1c3169ac8502 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:44.366168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22b0d3-34c5-4e3b-b1af-1c3169ac8502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:44.384350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-dc22b0d3-34c5-4e3b-b1af-1c3169ac8502 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:44.385582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dc22b0d3-34c5-4e3b-b1af-1c3169ac8502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:44.394015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 760/1520] 10.222.0.36 () {58 vars in 1057 bytes} [Mon May 23 10:49:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:44.401534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c454b3-019b-45c3-9424-a0dfee221ab5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:44.415159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c454b3-019b-45c3-9424-a0dfee221ab5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:44.420184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:44.420808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:44.421095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:44.421376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:44.430197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:44.431803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c958f-c8e8-4e8a-a4e5-35c34d84f131 service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:44.441360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 761/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:46.783477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044b74-c3ad-439f-b24d-c8fe368347ba None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:46.784015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044b74-c3ad-439f-b24d-c8fe368347ba None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:46.784015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044b74-c3ad-439f-b24d-c8fe368347ba None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:46.785386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:46.790265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440cfb8-5771-4dd5-a7ea-a94c2413c793 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:46.790681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440cfb8-5771-4dd5-a7ea-a94c2413c793 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:46.790833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440cfb8-5771-4dd5-a7ea-a94c2413c793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:46.808928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f440cfb8-5771-4dd5-a7ea-a94c2413c793 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:46.810149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f440cfb8-5771-4dd5-a7ea-a94c2413c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:46.817792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:46.968329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb044b74-c3ad-439f-b24d-c8fe368347ba None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:46.982869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb044b74-c3ad-439f-b24d-c8fe368347ba None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:46.988381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:46.989029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:46.989319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:46.989599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:46.998484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:47.000101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb26db2-b411-4709-9eac-6345c93ad946 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:47.010036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 762/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:47.867654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc403b-0233-41e7-938f-cd5b302d21b9 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:47.868214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc403b-0233-41e7-938f-cd5b302d21b9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:47.868214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc403b-0233-41e7-938f-cd5b302d21b9 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:47.869636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 10:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:47.874314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f49171-6e7c-4073-9b68-daead213ed14 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:47.874709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f49171-6e7c-4073-9b68-daead213ed14 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:47.874880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f49171-6e7c-4073-9b68-daead213ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:47.893563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e9f49171-6e7c-4073-9b68-daead213ed14 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:47.895062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e9f49171-6e7c-4073-9b68-daead213ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:47.903542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 10:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:48.053271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc403b-0233-41e7-938f-cd5b302d21b9 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:48.067494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc403b-0233-41e7-938f-cd5b302d21b9 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:48.073423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:48.074094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:48.074396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:48.074672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:48.082743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:48.084338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b20100-bf8b-4edd-a79a-3d23de636328 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:48.094733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 764/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:52.430014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:52.430527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:52.430527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:52.431906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:52.437081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d57aa-e088-43d6-84b6-2af9daddd330 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:52.437478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d57aa-e088-43d6-84b6-2af9daddd330 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:52.437632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d57aa-e088-43d6-84b6-2af9daddd330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:52.456024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fd7d57aa-e088-43d6-84b6-2af9daddd330 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:52.457223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fd7d57aa-e088-43d6-84b6-2af9daddd330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:52.465591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:52.501091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:52.502797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:52.558852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7aae2-ad3e-46e2-8cbc-0f72747a5162 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:52.562395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:52.563027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:52.563310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:52.563648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:52.572849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:52.574314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cf10d-8f35-4b41-986b-b2258634f8f4 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:52.583537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 765/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:49:54.740206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9f2e-2723-4089-9024-90f7b5719429 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:54.740774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9f2e-2723-4089-9024-90f7b5719429 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:54.740774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db9f2e-2723-4089-9024-90f7b5719429 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:54.742769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:54.748003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8be5ec-4363-4640-b8c4-9cc805b13f82 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:54.748431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8be5ec-4363-4640-b8c4-9cc805b13f82 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:54.748568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8be5ec-4363-4640-b8c4-9cc805b13f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:54.766568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3e8be5ec-4363-4640-b8c4-9cc805b13f82 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:54.767794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3e8be5ec-4363-4640-b8c4-9cc805b13f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:54.775907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:54.811005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db9f2e-2723-4089-9024-90f7b5719429 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:54.821568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db9f2e-2723-4089-9024-90f7b5719429 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:54.825077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:54.825714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:54.826027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:54.826313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:54.835064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:54.836498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b32813-d722-4a57-8471-b8d1077fa92c service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:54.845782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 767/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:49:58.677448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:58.678016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:58.678016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:58.680052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:49:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:49:58.685477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbb6b-7066-4d0f-b828-919a1e94d5c2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:58.685919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbb6b-7066-4d0f-b828-919a1e94d5c2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:58.686043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dbb6b-7066-4d0f-b828-919a1e94d5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:58.703661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-799dbb6b-7066-4d0f-b828-919a1e94d5c2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:49:58.704876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-799dbb6b-7066-4d0f-b828-919a1e94d5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:58.713122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:49:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:49:58.983400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:49:58.985056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:49:59.042256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58897a78-459e-4cca-88df-8fb35bd7a723 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:49:59.045796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:49:59.046469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:49:59.046763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:49:59.047042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:49:59.056442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:49:59.057893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db552e9-4cbd-4c5e-9b5a-8722ef5b626d service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:49:59.062837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:49:59.067786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 768/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:49:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:50:01.270086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:50:01.270547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:50:01.270645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:50:01.272035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 10:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:50:01.277278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006cd2c-b242-4985-85d8-fc5d02479a5a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:50:01.277704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006cd2c-b242-4985-85d8-fc5d02479a5a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:50:01.277822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006cd2c-b242-4985-85d8-fc5d02479a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:50:01.295761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c006cd2c-b242-4985-85d8-fc5d02479a5a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:50:01.296951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c006cd2c-b242-4985-85d8-fc5d02479a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:50:01.305235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 10:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:50:01.572855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:50:01.574563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:50:01.582067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=94 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:50:01.588722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c377cd-ad12-49e6-8698-66a67d568774 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:50:01.592339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:50:01.592979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:50:01.593267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:50:01.593578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:50:01.603074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:50:01.604514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38e3e6-5df7-42c6-8d88-2be02c91e19b service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:50:01.613882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 770/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:50:11.290373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:50:11.609689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:34.348046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.349121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.349612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.380331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.382551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.397846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 770/1540] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.409007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.409605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.409605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.438025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.439261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.453738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 771/1541] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.458554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:34.472792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b01549-89fd-4574-b9b0-f23e6defcea4 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:34.482432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:34.483605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.484130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.484625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.485623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:34.490645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:34.492947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c864d-a43c-4d7a-9669-fdd8fb77bbf1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:34.508670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 771/1542] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:34.532631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:34.548073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93ab03-5762-469a-9ba3-35abe162fc85 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:34.562289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:34.563384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.563906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.564417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.565357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:34.567646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:34.582964 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46d9a-a541-4d44-afe3-1c8809f01589 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.583647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46d9a-a541-4d44-afe3-1c8809f01589 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.584878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd46d9a-a541-4d44-afe3-1c8809f01589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.617879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 772/1543] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:34.618997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9dd46d9a-a541-4d44-afe3-1c8809f01589 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.620371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9dd46d9a-a541-4d44-afe3-1c8809f01589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.632202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:34.738345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 772/1544] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.739852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eed15-d9a4-4c3c-867c-7d58b82a730a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.740149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eed15-d9a4-4c3c-867c-7d58b82a730a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.740511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83eed15-d9a4-4c3c-867c-7d58b82a730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.748912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d2e27-a415-4c09-a580-6257614ac158 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:34.760709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:34.761397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.761677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.761975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.762668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:34.763692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:51:34.763692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:51:34.763692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:51:34.764941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef066c02-45be-41cb-a799-5983325da008 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:34.775594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f83eed15-d9a4-4c3c-867c-7d58b82a730a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.776928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f83eed15-d9a4-4c3c-867c-7d58b82a730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.785266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 773/1545] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.786623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe9605-2a60-4aa3-afac-de334a58d62b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.786914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe9605-2a60-4aa3-afac-de334a58d62b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.787887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe9605-2a60-4aa3-afac-de334a58d62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.789119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 773/1546] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:51:34.790384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3994a2-3bcc-4758-8cb2-2b2859fb54d2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.790676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3994a2-3bcc-4758-8cb2-2b2859fb54d2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.790958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3994a2-3bcc-4758-8cb2-2b2859fb54d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.808779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-48fe9605-2a60-4aa3-afac-de334a58d62b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.810017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-48fe9605-2a60-4aa3-afac-de334a58d62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.814468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7c3994a2-3bcc-4758-8cb2-2b2859fb54d2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.815634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7c3994a2-3bcc-4758-8cb2-2b2859fb54d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.817795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 774/1547] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.823297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ecb4-38f2-48f8-81c0-73971ddc77ad None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.825286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ecb4-38f2-48f8-81c0-73971ddc77ad None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.825420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 774/1548] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.825516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1ecb4-38f2-48f8-81c0-73971ddc77ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.825603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3586-a64e-409d-89c8-bbead7906b59 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.825603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3586-a64e-409d-89c8-bbead7906b59 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.825760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3586-a64e-409d-89c8-bbead7906b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.844657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9b9f3586-a64e-409d-89c8-bbead7906b59 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.844943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4ea1ecb4-38f2-48f8-81c0-73971ddc77ad None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.845878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9b9f3586-a64e-409d-89c8-bbead7906b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.846138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4ea1ecb4-38f2-48f8-81c0-73971ddc77ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.854352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 775/1549] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.858443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 775/1550] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.858443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.858443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.858443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.859091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4351-523e-43c4-8b96-4c07aebccb4c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.859450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4351-523e-43c4-8b96-4c07aebccb4c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.859740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4351-523e-43c4-8b96-4c07aebccb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.875388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.876590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.877771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2d5e4351-523e-43c4-8b96-4c07aebccb4c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.878993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2d5e4351-523e-43c4-8b96-4c07aebccb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.884812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 776/1551] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.885711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:34.893672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 776/1552] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.895318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd7589-b035-4524-8893-5583eb33f007 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.895725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd7589-b035-4524-8893-5583eb33f007 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.896078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd7589-b035-4524-8893-5583eb33f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.901349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888ef1-6b66-4285-9397-df868cf6d314 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:34.906364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:34.907027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.907312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.907589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.908341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:34.910133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472fd85-fd4b-4ab5-b2ad-23c23a00b0fb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:34.915160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-93cd7589-b035-4524-8893-5583eb33f007 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.916357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-93cd7589-b035-4524-8893-5583eb33f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.923504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 777/1553] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:34.923504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.923504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.923504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.924305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 777/1554] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.925532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0d8-94ca-43e1-afb0-07453e5a864b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.925824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0d8-94ca-43e1-afb0-07453e5a864b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.926139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea0d8-94ca-43e1-afb0-07453e5a864b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.942448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.943132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b5eea0d8-94ca-43e1-afb0-07453e5a864b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.943705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.944356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b5eea0d8-94ca-43e1-afb0-07453e5a864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.951404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 778/1555] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.955379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:34.955820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 778/1556] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.957108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a3ef-a47b-4dd7-9677-113778f66eeb None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.957400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a3ef-a47b-4dd7-9677-113778f66eeb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.957688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92a3ef-a47b-4dd7-9677-113778f66eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.962351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d786b4-8449-4046-871a-2a5e7c2360ff None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:34.966842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:34.967475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.967758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.968550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.969274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:34.969577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:34.971211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457f2cb-82b2-478b-9b24-25184459de2b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:34.976625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-db92a3ef-a47b-4dd7-9677-113778f66eeb None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:34.977813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-db92a3ef-a47b-4dd7-9677-113778f66eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:34.983086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 779/1557] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:34.989205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 779/1558] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:34.991387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.991533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:34.991674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.991770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:34.991950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:34.992045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.009254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:35.009471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:35.010415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:35.010658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:35.018182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 780/1559] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:35.018949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.019337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 780/1560] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:35.020395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.029202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b941b0b-67a3-4cda-9d34-0fcab4563dce None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.031268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0ffae-fae6-44ee-8818-0ca13600e072 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.033808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.034486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.034775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.035051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.035766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.035883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.036610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.037077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.038000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.038129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.039239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.039625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.040802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 10:51:35.040802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:51:35.040802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:51:35.041680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.048189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 781/1561] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.048982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.053857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 781/1562] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.054692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.064153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7074ce-d2a5-44f5-95e1-d9daf37d32e5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.064600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0507c88-e010-484c-ac77-91ab1fe7863c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.069173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.069643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.069805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.070130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.070316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.070423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.070603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.070882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.071106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.071433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.071599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.071915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.073004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.073498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.085745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 782/1563] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.091290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 782/1564] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.091290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.091523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.100838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db08fad-2474-4266-83f1-59404104e4ae admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.102822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7275-aff2-4314-8826-28ef7940533e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.105359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.106014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.106306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.106584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.107273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.107397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.107958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.108239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.108344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:51:35.108344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:51:35.108344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:51:35.108600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.109312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.109503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.109607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.111203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.122852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 783/1565] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.123637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.133722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30223a-ed7e-4c5d-ab0c-245412ec6953 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.139229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.139956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.140312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.140663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.141443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.141838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.146631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 783/1566] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.147287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.147385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.157325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 784/1567] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.158308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.171823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917301b-725e-4c59-a5f9-5099889f948c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.184271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.191710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.214496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71d76c-b922-4f6f-92b9-84c0574c2c2b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.219374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.220062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.220349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.220632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.221354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:35.221660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.223319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.234402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.235585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.235867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.236144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] PATH_INFO: `/v3/projects/2707ec87254b4e55b636f5a5ac143858/users/ba02be702029458c837072cf0490f600/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.237053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 784/1568] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:35.237918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.248290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be5be7-44e8-4b7c-937a-afe699087f69 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.252925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.253078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2707ec87254b4e55b636f5a5ac143858, user_id=ba02be702029458c837072cf0490f600, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.253572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.253857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.254172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.254868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.255677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.257386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.269604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 785/1569] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:35 2022] PUT /identity/v3/projects/2707ec87254b4e55b636f5a5ac143858/users/ba02be702029458c837072cf0490f600/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:35.273831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.292277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 785/1570] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.293367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.335686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6ccf0-ab81-49ea-b97c-ab6085a0f11b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.352881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c95d-5651-422c-8566-3afd4972349a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.357472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.358155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.358441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.358710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.359399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.361316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.388685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 786/1571] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.389492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.391669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.392722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.393004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.393336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.394322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.396543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.421837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 786/1572] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.422790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.454311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a770b-1c97-4998-98a1-8c6c1498e6eb admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.464176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.485256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2cb1e-6f45-4085-8549-2842c0c28147 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.485770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.490952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.492868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.493156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.493433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.494152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.496088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.513564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.514304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.514574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.514852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.515542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.517017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 787/1573] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.517597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.517799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.534009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb041436-844d-4999-942a-096ad76b2fca admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.539886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.540572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.540857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.541131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.541824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.543794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908d876-4d1a-443e-a6d7-2449075abcc7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.545060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 787/1574] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.545871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.574938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 788/1575] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.574938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.574938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.574938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.598548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] MFA Rules not processed for user `ba02be702029458c837072cf0490f600`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:35.599825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:35.609257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d4842-1dfd-4033-8a84-4d9ede778b20 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.664488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 789/1576] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:35.664815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.665387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.665496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.665805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.666160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.666849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.669340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.676000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40407165-d8c3-4522-96a2-15a36ea97ca0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.681501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.682169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.682492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.682881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.683619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.685460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.702698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 788/1577] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/projects => generated 369 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.703681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.714201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 790/1578] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.715016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.724466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99f291-62bd-4204-a5c8-c9d6c4bf4a64 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.759233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.773381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22e548-e8f2-45c5-9226-e66c1ef157c7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.778692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.779356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.779637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.779918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.780600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.782597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.782749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.783439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.783729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.784059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.784702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.786851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.809211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 791/1579] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.810100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.818281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 789/1580] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.818281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.820168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc914ca-da73-460e-a5bf-25dbdbfa4647 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.826327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ee0f8-4b41-425b-a0a9-3d6565945a0b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.828383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.829124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.829416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.829698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.830466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.831235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.831935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.832076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.832270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.832495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.833178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.835136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853643f-e43c-4b66-8a48-6557f683bee1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.837382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:35.853749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 792/1581] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.853749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.857459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 790/1582] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.862004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da12f4aa-0d23-4930-b024-99d90ec5137a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.863462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31a627-7e88-4e74-8030-ba114887c3d1 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.863758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31a627-7e88-4e74-8030-ba114887c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.864040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31a627-7e88-4e74-8030-ba114887c3d1 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.866538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 791/1583] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 23 10:51:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:35.867364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31a627-7e88-4e74-8030-ba114887c3d1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.867488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.868036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.868325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.868605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.869295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.871250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.884586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31a627-7e88-4e74-8030-ba114887c3d1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.889226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.889883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.890205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.890486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.891178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.894475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 793/1584] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.894475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.894938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.909310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d74146-0ebc-4365-9206-a267885e135d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.914534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.914785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 792/1585] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/users => generated 285 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:35.915741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.916022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.916304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.917043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.917144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.918733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.928558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51402be0-68e6-4c33-aa3d-3707e00b6d50 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.929864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 794/1586] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.933219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.933753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.933871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.934187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.934463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.935179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.936821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.949913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 793/1587] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.949913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.950221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354900e-eb6f-42fa-8412-0fab96aaa6a9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.952928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.953592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.953876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.954188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.954910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.957664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaec8e2-4d68-4039-b75f-428b4f64d3c1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.957898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.962505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.963159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.963447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.963720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.964508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.966141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262804c-729f-49d4-b7be-ae1960220305 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.972619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 795/1588] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.973380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:35.976831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 794/1589] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:35.978702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.978702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.978925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.984067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a876-f857-4cd5-bb50-f676fe740ec9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:35.988860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:35.989513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:35.989795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:35.990101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:35.991400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:35.993016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:35.997071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:35.998563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.004164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 796/1590] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.005053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.015386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62f0ad-59ca-4902-83c0-a8168334064c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.021204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.021893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.022216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.022492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.023213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.025496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.036960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 797/1591] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.038460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.048711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f753ff-e400-4c9d-9476-53cf208ee6ff admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.055118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.056030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.056612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.057050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.057969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.060023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.072624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 798/1592] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.073482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.080359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 795/1593] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 23 10:51:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.081308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.083658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb11f9-a3fa-4b5d-8efa-f5b3a871658a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.088542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.089244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.089531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.089807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.090571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.091990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7b410-5c7e-4b6e-be8e-a1a75d9e6849 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.092182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.096535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.097190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.097478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.097753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.098509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.100625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.106199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 799/1594] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.106199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.114083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 796/1595] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.114083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.116557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2db7f-7c66-4fd8-9af6-73de19a1d5fe admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.122323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d138c-78c0-49e9-ad50-a52b82b4765c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.125434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.126144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.126425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.126727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.127474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.127603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.128417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.128704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.128986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] PATH_INFO: `/v3/projects/e1d35cdfa73449c2bd040f966d9f27a2/users/4b61e9897f8e4c8ab2fd8cd6b5244ca5/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.129128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.142947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 800/1596] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.142947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.144061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d35cdfa73449c2bd040f966d9f27a2, user_id=4b61e9897f8e4c8ab2fd8cd6b5244ca5, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.145390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:51:36.145390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:51:36.145390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:51:36.148174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.152824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5354234-4e80-415f-8efa-43d61b3c1cf4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.157509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.158193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.158479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.158758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.159560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.161091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.161622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 797/1597] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/e1d35cdfa73449c2bd040f966d9f27a2/users/4b61e9897f8e4c8ab2fd8cd6b5244ca5/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.163121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.175540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 801/1598] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.176535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.185869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ab49-49dd-4134-af47-bcb512abc274 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.211997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:36.233003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55638-e8e8-47ff-a52a-1cb2d518cb5d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.237917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.238692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.239035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.239373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.240886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.242628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.247903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.248567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.248912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.249123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.249843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.251524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.255845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 798/1599] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.256617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.266295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 802/1600] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:36.266295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.276564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36297bd8-1d99-4862-b1d6-44086e91334a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.278639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.279504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.279785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.280057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] PATH_INFO: `/v3/projects/2c669b26bf1845a6a3b35eb551ebcb98/users/7433cc7ed66f4e609332c56c936487e8/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.298150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c669b26bf1845a6a3b35eb551ebcb98, user_id=7433cc7ed66f4e609332c56c936487e8, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.298150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.312527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 803/1601] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/2c669b26bf1845a6a3b35eb551ebcb98/users/7433cc7ed66f4e609332c56c936487e8/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.313558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.340923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:36.341202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426323f3-f452-496b-916a-19b05e650cba admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.384433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553b85b-e6a2-47a2-960f-b669eef93bdb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.389495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.390416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.390913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.391051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] PATH_INFO: `/v3/projects/9d2278c536e14f7e96a3b01a18cd19ad/users/448ee046d0bc462c8eeeb839632904f0/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.408763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2278c536e14f7e96a3b01a18cd19ad, user_id=448ee046d0bc462c8eeeb839632904f0, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.409845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.410533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.410818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.411096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.411576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.427458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 804/1602] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/9d2278c536e14f7e96a3b01a18cd19ad/users/448ee046d0bc462c8eeeb839632904f0/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.428279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.477197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.478868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfb100-70b2-4631-a7e6-e8fc192effd8 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.488404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 799/1603] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:36.489275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe4e8c-49be-478c-941d-a64efcc24d87 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.489824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.490216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.490615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.494385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.495352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.495703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.496042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] PATH_INFO: `/v3/projects/00d4a5543a3f41a6b3dab9250c0ce9be/users/8e98c8ceac6b47079423b45685c5ba3e/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.508022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.509320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.510744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d4a5543a3f41a6b3dab9250c0ce9be, user_id=8e98c8ceac6b47079423b45685c5ba3e, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.513470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.517451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 800/1604] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.518371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.535704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 805/1605] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/00d4a5543a3f41a6b3dab9250c0ce9be/users/8e98c8ceac6b47079423b45685c5ba3e/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.535884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ce3c6-20f8-4749-9be5-292ed792c082 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.536664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.541278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:36.600968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.601811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.602187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.602531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] PATH_INFO: `/v3/projects/3d45f08040d949bd8949e3d1faab25c4/users/a1fc33c4ee0c4b30a862f6685817bdf4/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.603715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0245-a80e-4544-b75f-7d286d619617 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.609291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.610277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.610639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.610987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] PATH_INFO: `/v3/projects/67ae1bdc29c44536a14d2918d4e1f61e/users/f60d2e6bddb64ad39ddaf8325ca1cff3/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.620881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d45f08040d949bd8949e3d1faab25c4, user_id=a1fc33c4ee0c4b30a862f6685817bdf4, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.623730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.626360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ae1bdc29c44536a14d2918d4e1f61e, user_id=f60d2e6bddb64ad39ddaf8325ca1cff3, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.629273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd947ee-004d-40cf-a13d-d455017db493 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.639008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 801/1606] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/3d45f08040d949bd8949e3d1faab25c4/users/a1fc33c4ee0c4b30a862f6685817bdf4/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.643074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.645304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 806/1607] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/67ae1bdc29c44536a14d2918d4e1f61e/users/f60d2e6bddb64ad39ddaf8325ca1cff3/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.647534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05abb0-95d9-43a2-9ae8-c31b05a75522 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.647827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05abb0-95d9-43a2-9ae8-c31b05a75522 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.648106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05abb0-95d9-43a2-9ae8-c31b05a75522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.666919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ac05abb0-95d9-43a2-9ae8-c31b05a75522 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.668147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ac05abb0-95d9-43a2-9ae8-c31b05a75522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.673695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:36.720523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654fb3b-691f-407c-98b2-03c8b4d0626e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.739944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 807/1608] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.741449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414ed8-c3b4-43d1-90c8-8a687c71aac6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.741741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414ed8-c3b4-43d1-90c8-8a687c71aac6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.742045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54414ed8-c3b4-43d1-90c8-8a687c71aac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.772373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-54414ed8-c3b4-43d1-90c8-8a687c71aac6 None None] MFA Rules not processed for user `448ee046d0bc462c8eeeb839632904f0`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.773585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-54414ed8-c3b4-43d1-90c8-8a687c71aac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.781058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.781969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.782365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.782650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] PATH_INFO: `/v3/projects/bfc709155cdb4a34a24fc5bd463716a5/users/c5a42fb25848435c85ec4c3c73d97b80/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.797302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfc709155cdb4a34a24fc5bd463716a5, user_id=c5a42fb25848435c85ec4c3c73d97b80, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.800085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.813503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 802/1609] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:36 2022] PUT /identity/v3/projects/bfc709155cdb4a34a24fc5bd463716a5/users/c5a42fb25848435c85ec4c3c73d97b80/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:36.823944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.840084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 808/1610] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.841576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25a6d-dcd3-4ee2-9a1e-8131270137c5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.841993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25a6d-dcd3-4ee2-9a1e-8131270137c5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.842385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f25a6d-dcd3-4ee2-9a1e-8131270137c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.846202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:36.867095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-38f25a6d-dcd3-4ee2-9a1e-8131270137c5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.868313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-38f25a6d-dcd3-4ee2-9a1e-8131270137c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.882777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c3cb-32a4-4655-ada5-f5d7e1ef11cc admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.886639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.887329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.887613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.887887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.927968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 809/1611] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.929452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2746a-3a96-44f4-97ff-7610d9c4bb6b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.929832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2746a-3a96-44f4-97ff-7610d9c4bb6b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.930304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2746a-3a96-44f4-97ff-7610d9c4bb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.947056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-94e2746a-3a96-44f4-97ff-7610d9c4bb6b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.948255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-94e2746a-3a96-44f4-97ff-7610d9c4bb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.950027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:36.951484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc346c9-fb63-4feb-b06b-b1ba16a83929 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:36.956115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 810/1612] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:36.957151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.957436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.957713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.965466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 803/1613] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:36.966965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982418b-1412-497c-ad79-fff6fae62a57 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.967320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982418b-1412-497c-ad79-fff6fae62a57 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.967611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982418b-1412-497c-ad79-fff6fae62a57 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.969174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 804/1614] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 23 10:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:36.970090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982418b-1412-497c-ad79-fff6fae62a57 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:36.983060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982418b-1412-497c-ad79-fff6fae62a57 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:36.985605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] MFA Rules not processed for user `f60d2e6bddb64ad39ddaf8325ca1cff3`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:36.987013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:36.987134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:36.988044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:36.988415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:36.988620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:36.999591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.001205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5060f90-3783-4d7a-89e7-190abbfce5bb service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.011401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 805/1615] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.012668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.012957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.013241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.015873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:37.050470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] MFA Rules not processed for user `c5a42fb25848435c85ec4c3c73d97b80`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:37.052136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:37.070744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 811/1616] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:37.071809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.138752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 806/1617] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:37.139611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.151253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69afde4-48c1-4182-b04a-78d2c17617a5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.153070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628c094-245c-4ad3-8ddc-cca19804093e None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.156412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.157232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.158012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.158123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.158239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.158516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.158811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.159147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.171423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.173111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.187345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 807/1618] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.188498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.197654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2700f740-11f1-42e7-8f30-2f20191a0d1a service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.204051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.204051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.204051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.204331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.216383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.218107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8c4cc-c929-4d73-b74d-ed1990cb4a5d service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.229274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 808/1619] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.234301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1d8c-2182-4aad-afd2-87218bd1dda5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.234622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1d8c-2182-4aad-afd2-87218bd1dda5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.234909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1d8c-2182-4aad-afd2-87218bd1dda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.247364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.248928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.257654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-efff1d8c-2182-4aad-afd2-87218bd1dda5 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:37.259397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-efff1d8c-2182-4aad-afd2-87218bd1dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:37.260342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 812/1620] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.261245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.276456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:37.281615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2dff-431b-4c80-9791-08723f9b6220 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.290016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.290814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.291179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.291941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.292765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.295776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271cca4-10dc-459a-a108-e4b3af18bd5e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.340447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 813/1621] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:37 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:37.344008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee48f7-9d9f-4115-8e73-742f76659755 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.344271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee48f7-9d9f-4115-8e73-742f76659755 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.344557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee48f7-9d9f-4115-8e73-742f76659755 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.346716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 814/1622] 10.222.0.36 () {54 vars in 961 bytes} [Mon May 23 10:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:37.347848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee48f7-9d9f-4115-8e73-742f76659755 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.358831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 809/1623] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:51:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:37.381998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.382439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.383084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.432540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:37.434309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:37.452434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:37.463041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee48f7-9d9f-4115-8e73-742f76659755 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.471625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.472437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.473320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.488712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.489742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.493276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.543288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 815/1624] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:37 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:37.544457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.612647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 810/1625] 10.222.0.36 () {58 vars in 1058 bytes} [Mon May 23 10:51:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:37.613680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.630693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d097e0-f728-44f4-9f68-bf5337a73cc9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.640935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.641874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.642397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.642902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.643916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.647319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.656789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79300011-ce7c-4d63-a613-35a7bacf65fb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.666853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.667574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.667857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.668190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.676331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 811/1626] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:37.676331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.696495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.697976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.707080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2dc89d-be1d-48f7-ab0a-66d6925e0ef9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.722878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 816/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.724239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.726006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.726757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.727053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.727731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.737631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c9bb8-d2c2-4c72-9388-43d20c85d674 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.743812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.745512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.746058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.746575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] PATH_INFO: `/v3/projects/4911e138ddc94697aaf598d2dd43d853/users/6ce746b1ec8f46ca8d01f93363646984/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.773173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4911e138ddc94697aaf598d2dd43d853, user_id=6ce746b1ec8f46ca8d01f93363646984, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.778325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.809654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 817/1628] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:37 2022] PUT /identity/v3/projects/4911e138ddc94697aaf598d2dd43d853/users/6ce746b1ec8f46ca8d01f93363646984/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:37.813528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:37.875881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.877559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:37.889552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 812/1629] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:37.974889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73011b84-f5ff-4990-97a5-7444c17e31fa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:37.984023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:37.986811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:37.987105 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:37.987396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:37.988360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:37.990757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce854-da87-484e-a4d3-cdc355b38ace admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:38.010546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 818/1630] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:38.011846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:38.025736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b45c5-adac-4d39-a931-b9c839327fd2 service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:38.043399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:38.044495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.045029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.045556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] PATH_INFO: `/v3/projects/4911e138ddc94697aaf598d2dd43d853/users/6ce746b1ec8f46ca8d01f93363646984/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.054631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4911e138ddc94697aaf598d2dd43d853, user_id=6ce746b1ec8f46ca8d01f93363646984, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:38.058427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d11266-8e72-4840-bf94-5561859e530e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:38.086273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 813/1631] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:38 2022] PUT /identity/v3/projects/4911e138ddc94697aaf598d2dd43d853/users/6ce746b1ec8f46ca8d01f93363646984/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:38.099163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.101011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.101521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.180693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] MFA Rules not processed for user `6ce746b1ec8f46ca8d01f93363646984`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:38.183150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:38.312271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcadf1-3ae0-4232-a6ef-04fe39cdf847 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.313294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcadf1-3ae0-4232-a6ef-04fe39cdf847 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.314075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcadf1-3ae0-4232-a6ef-04fe39cdf847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.356221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:38.384925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-05dcadf1-3ae0-4232-a6ef-04fe39cdf847 None None] MFA Rules not processed for user `4b61e9897f8e4c8ab2fd8cd6b5244ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:38.386391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-05dcadf1-3ae0-4232-a6ef-04fe39cdf847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:38.400835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:38.403927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 819/1632] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:38.433397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:38.566392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 814/1633] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:51:38.568532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda37da-9e60-4952-8039-ada66d590d68 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.568786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda37da-9e60-4952-8039-ada66d590d68 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.569083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda37da-9e60-4952-8039-ada66d590d68 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.573323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 815/1634] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 23 10:51:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:38.574737 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06bc4-b912-4354-9389-53e8d1648043 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.575033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06bc4-b912-4354-9389-53e8d1648043 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.575316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06bc4-b912-4354-9389-53e8d1648043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.607181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e79bde-6fda-42f9-a785-ef09ac01a99d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:38.613844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bba06bc4-b912-4354-9389-53e8d1648043 None None] MFA Rules not processed for user `4b61e9897f8e4c8ab2fd8cd6b5244ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:38.616004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:38.617877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.618523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bba06bc4-b912-4354-9389-53e8d1648043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:38.620058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.620450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.643331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 816/1635] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:38.649836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.650166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.651633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.656136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:38.657844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:38.687281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 820/1636] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:51:38.687993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:38.732616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:38.734558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:38.867227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d757fe-1eee-411b-ad69-29fbda85421c service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:38.876507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:38.877191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.877470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.877830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.895249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:38.897606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:38.910355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 817/1637] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:38.911365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:38.923037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 821/1638] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:38.924002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:38.940400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314ed76-da88-4a34-a2ce-37f7caf28d2b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:38.942989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47097ba-486d-4417-b1f5-75ee078feb2e service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:38.944867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:38.945523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.945810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.946109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.952736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:38.953409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:38.953699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:38.953992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:38.968725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:38.970310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840800cd-6c0c-4880-bea4-66075cce0a39 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:39.008686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 818/1639] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:39.097435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:39.100983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe4172-6578-4317-9efb-23305c7ab95e service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:39.133857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 822/1640] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:39.658596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457e3b-858e-4146-b374-4329e0eba800 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:39.659979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457e3b-858e-4146-b374-4329e0eba800 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:39.660887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61457e3b-858e-4146-b374-4329e0eba800 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:39.663400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 819/1641] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 23 10:51:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:39.669212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a4b07-d879-4eec-b511-492ae5bfc032 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:39.669725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a4b07-d879-4eec-b511-492ae5bfc032 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:39.669820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a4b07-d879-4eec-b511-492ae5bfc032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:39.706713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:39.730470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e74a4b07-d879-4eec-b511-492ae5bfc032 None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:39.732135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e74a4b07-d879-4eec-b511-492ae5bfc032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:39.813287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 823/1642] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:51:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:39.834151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61457e3b-858e-4146-b374-4329e0eba800 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:39.851065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61457e3b-858e-4146-b374-4329e0eba800 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:39.856646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:39.857597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:39.858179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:39.858723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:39.869463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:39.872109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff46ea1-52ec-47fd-8901-815572f09c79 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:39.885319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 820/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:41.141695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0c5e-c16f-4a33-a7cb-7da89368c4a7 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:41.142700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0c5e-c16f-4a33-a7cb-7da89368c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:41.143222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0c5e-c16f-4a33-a7cb-7da89368c4a7 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:41.145767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 824/1644] 10.222.0.36 () {54 vars in 960 bytes} [Mon May 23 10:51:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:41.151704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0bfe-14a1-4892-89ee-fa95a77760b5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:41.152308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0bfe-14a1-4892-89ee-fa95a77760b5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:41.152308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e0bfe-14a1-4892-89ee-fa95a77760b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:41.184528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7e1e0bfe-14a1-4892-89ee-fa95a77760b5 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:41.186444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7e1e0bfe-14a1-4892-89ee-fa95a77760b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:41.334127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 821/1645] 10.222.0.36 () {58 vars in 1057 bytes} [Mon May 23 10:51:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:41.342949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a0c5e-c16f-4a33-a7cb-7da89368c4a7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:41.371386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a0c5e-c16f-4a33-a7cb-7da89368c4a7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:41.384453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:41.387978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:41.388471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:41.388781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:41.391731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:41.407333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:41.409360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:41.436618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 825/1646] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:46.586118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b853-9ad7-4298-bb25-21a9e1415aad None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:46.586654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b853-9ad7-4298-bb25-21a9e1415aad None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:46.586654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9b853-9ad7-4298-bb25-21a9e1415aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:46.595359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:46.643392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-11a9b853-9ad7-4298-bb25-21a9e1415aad None None] MFA Rules not processed for user `a1fc33c4ee0c4b30a862f6685817bdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:46.645112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-11a9b853-9ad7-4298-bb25-21a9e1415aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:46.858918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 822/1647] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:46.905120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:46.928996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0242a8-be70-4d47-9ea7-b3e1ac6cf91e service placement] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:46.943729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:46.946929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:46.950889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:46.954040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:46.971382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:46.975949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe66ff0-4c59-4120-80e5-4dee65ad4845 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:47.003212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 826/1648] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:49.869141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:49.869141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:49.869574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:49.907403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:49.909268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:49.987031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:49.987584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:49.987868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.079065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] MFA Rules not processed for user `8e98c8ceac6b47079423b45685c5ba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:50.080801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 823/1649] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:51:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:51:50.082411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:50.246612 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:50.334404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 827/1650] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:50.347417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:50.385806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40f841-55b7-468c-b3e0-05574cc75ebd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:50.425134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:50.425817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:50.426133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:50.430009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.448272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:50.449921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:50.461034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba3530-aa69-4f00-8841-2cefff0ae3e9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:50.470917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 828/1651] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:51:50.471956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:50.479120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:50.535440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:50.539445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:50.543836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:50.544889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] PATH_INFO: `/v3/users/c5a42fb25848435c85ec4c3c73d97b80` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.551807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a42fb25848435c85ec4c3c73d97b80)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:50.552826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: WARNING py.warnings [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:51:50.552826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: warnings.warn(msg) May 23 10:51:50.552826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: May 23 10:51:50.557473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:50.583159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:50.597576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a4ffd-031a-4c01-8534-361dfcb58df5 service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:50.621097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:50.623305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:50.623905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:50.627388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.643778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:50.645670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb26508-bbde-4ffb-bf55-c30c6becb144 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:50.668359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 829/1652] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:51:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:51:50.712625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:50.737429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a42fb25848435c85ec4c3c73d97b80'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:50.745148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:50.745610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:50.746170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.827007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a42fb25848435c85ec4c3c73d97b80'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:50.830768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] MFA Rules not processed for user `7433cc7ed66f4e609332c56c936487e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:50.834813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a42fb25848435c85ec4c3c73d97b80'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:50.836094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:50.837026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a42fb25848435c85ec4c3c73d97b80'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:50.853180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dc64ec1c-0523-4c11-9540-85c5643b71ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a42fb25848435c85ec4c3c73d97b80'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:50.913545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 824/1653] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:51:50 2022] DELETE /identity/v3/users/c5a42fb25848435c85ec4c3c73d97b80 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:50.917914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:50.921789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:50.922409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:50.961540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 830/1654] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:51.000380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:51.007456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:51.170228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:51.173194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:51.265147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19eef4-f172-42b9-830a-db24ad8d041b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:51.279811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 825/1655] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:51.291042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:51.296541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:51.547715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6899008-faf7-44fe-bd98-90b55bf1f3e6 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:51.573252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:51.575150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:51.575819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:51.576094 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:51.576376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:51.579686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:51.580127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:51.580411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:51.623548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:51.627529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:51.693662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 826/1656] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:51.884273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:51.893500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825c25f-e56f-4a96-9ecd-eed1de032f48 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:51.951858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 831/1657] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:52.556979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:52.558738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:52.591182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf631-ed96-4a4b-855d-c007060fc6af service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:52.623381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:52.623381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:52.623381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:52.623381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] PATH_INFO: `/v3/users/ba02be702029458c837072cf0490f600` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:52.623846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba02be702029458c837072cf0490f600)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:52.633664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:52.803290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba02be702029458c837072cf0490f600'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:52.813522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:52.845194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba02be702029458c837072cf0490f600'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:52.846323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba02be702029458c837072cf0490f600'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:52.847833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba02be702029458c837072cf0490f600'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:52.862470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba02be702029458c837072cf0490f600'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:52.885565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 827/1658] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:51:52 2022] DELETE /identity/v3/users/ba02be702029458c837072cf0490f600 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:52.887812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2500-d46f-4412-afed-e1e2fb50bf5c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:52.888890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2500-d46f-4412-afed-e1e2fb50bf5c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:52.889339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b2500-d46f-4412-afed-e1e2fb50bf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:52.936465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-dc5b2500-d46f-4412-afed-e1e2fb50bf5c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:52.946713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dc5b2500-d46f-4412-afed-e1e2fb50bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:52.992487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:53.047939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:53.050827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:53.294348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 832/1659] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:53.299802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15241-e794-4ef7-8c32-610f527f1916 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:53.300248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15241-e794-4ef7-8c32-610f527f1916 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:53.300344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd15241-e794-4ef7-8c32-610f527f1916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:53.413279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bbd15241-e794-4ef7-8c32-610f527f1916 None None] MFA Rules not processed for user `a1fc33c4ee0c4b30a862f6685817bdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:53.420236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bbd15241-e794-4ef7-8c32-610f527f1916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:53.430669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98c64a-2a78-4e94-89fc-817472d1b764 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:53.454673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:53.455684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:53.467244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:53.467244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] PATH_INFO: `/v3/projects/bfc709155cdb4a34a24fc5bd463716a5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:53.467244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfc709155cdb4a34a24fc5bd463716a5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:53.467244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:53.562491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 833/1660] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:53.564427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:53.568647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:53.569267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:53.617075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:53.619198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:53.790583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 834/1661] 10.222.0.73 () {58 vars in 1015 bytes} [Mon May 23 10:51:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:53.791795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:53.794732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:53.855963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525818a2-6849-4035-b0c1-993ef6dba8cf None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:53.879654 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:53.882009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:53.883971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:53.884882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:53.917850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:53.922627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:53.988297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 835/1662] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:54.015159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:54.061842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166274f-3fb0-4ca0-b0d8-a5b236a29ab1 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:54.095926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:54.097141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:54.104728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:54.106024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:54.134605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfc709155cdb4a34a24fc5bd463716a5', 'request_id': 'req-425dc078-81cb-4250-aa26-7010c64eaac4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:54.145859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:54.155817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfc709155cdb4a34a24fc5bd463716a5', 'request_id': 'req-425dc078-81cb-4250-aa26-7010c64eaac4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:54.157767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf161c-595c-46d3-9a4f-4fc521866ff6 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:54.179091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfc709155cdb4a34a24fc5bd463716a5', 'request_id': 'req-425dc078-81cb-4250-aa26-7010c64eaac4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:54.204485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 836/1663] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:54.225592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] The token cache is being invalidate because project bfc709155cdb4a34a24fc5bd463716a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:51:54.225900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:54.230227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-425dc078-81cb-4250-aa26-7010c64eaac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:54.241442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 828/1664] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:51:53 2022] DELETE /identity/v3/projects/bfc709155cdb4a34a24fc5bd463716a5 => generated 0 bytes in 1190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:51:54.243251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:54.243762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:54.244436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:54.315582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:54.319655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:54.465989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:54.533021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 837/1665] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:54.543169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:54.543633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:54.544075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:54.635634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:54.640323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:54.671978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 829/1666] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:54.672533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:54.698614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f0940-608d-4bdb-b7fb-c89547339d6c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:54.720115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:54.722416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:54.722944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:54.723062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:54.727635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:54.727966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:54.729717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:54.779581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:54.801579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 838/1667] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:54.807540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb3423-f05b-417b-956e-326e0793953a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:54.835467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:54.836286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:54.836693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:54.838792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:54.838991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:54.847854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:54.933875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 830/1668] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:54 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:54.936813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:54.987085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734553fc-e111-4079-9965-da5c51c5fa0f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:55.191986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:55.193675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:55.194371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:55.195201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:55.196131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:55.199388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:55.248505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:55.249290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 839/1669] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:54 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:55.268322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427aac5-d036-4a6b-9915-0dfe0fc4307f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:55.310770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:55.311643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:55.312062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:55.312434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:55.313243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:55.327578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:55.381352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 831/1670] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:55.384412 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:55.397677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3021d82-a787-451d-b12b-84ce59ff6623 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:55.416365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:55.417220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:55.417599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:55.419233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:55.421792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:55.424559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80222e40-7975-43de-892f-e5271357d11e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:55.470283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 840/1671] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:55.470726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:55.484022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1c10f-05c5-42d4-87a8-221dc10ebda7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:55.495060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:55.501054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:55.504371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:55.504965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] PATH_INFO: `/v3/projects/d8013fb326a54588b6fc29be5476b99d/users/297487ae0a294490b56a967ce665e63b/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:55.525705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8013fb326a54588b6fc29be5476b99d, user_id=297487ae0a294490b56a967ce665e63b, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:55.530042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:55.545287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:55.553611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 832/1672] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:55 2022] PUT /identity/v3/projects/d8013fb326a54588b6fc29be5476b99d/users/297487ae0a294490b56a967ce665e63b/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:55.565472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:55.567409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:55.568458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:55.625534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] MFA Rules not processed for user `297487ae0a294490b56a967ce665e63b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:55.627208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:55.893620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 841/1673] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:55.898892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:56.167618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13911c-a9ea-44f7-a3b0-0822e02197ae admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:56.179752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:56.180448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:56.180845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:56.181036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:56.186615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:56.190490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:56.240209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:56.250633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 833/1674] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:55 2022] POST /identity/v3/projects => generated 383 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:56.252281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493972a-3b98-404e-b774-f97abfdc0d74 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:56.403789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:56.436744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:56.437644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:56.437934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:56.438234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:56.439468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:56.449571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:56.534415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 842/1675] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:56 2022] POST /identity/v3/users => generated 298 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:51:56.538844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:56.565781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d282741-22a6-4f66-8fcd-50479bae8590 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:56.590502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:56.591193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:56.591482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:56.591764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:56.592512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:56.607208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:56.635689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 834/1676] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:51:56.639034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:56.673679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608ef5c-98dc-47fb-ab0b-c2d25b277f5f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:56.712628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:56.713636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:56.716134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:56.716461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] PATH_INFO: `/v3/projects/bc60f187260348f1a28df37b3ca5a973/users/9f906926289346ec9c3864db6f76eca5/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:56.783937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc60f187260348f1a28df37b3ca5a973, user_id=9f906926289346ec9c3864db6f76eca5, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:56.785197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:56.818817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 843/1677] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:56 2022] PUT /identity/v3/projects/bc60f187260348f1a28df37b3ca5a973/users/9f906926289346ec9c3864db6f76eca5/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:56.820319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:56.839650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5bb6-bd8c-47b1-a7b8-3de9535ece2f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:57.044961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:57.046100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.046610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.047084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.049300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:57.057209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eef7af-92b8-47da-a5d8-8c25ad255866 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:57.082767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 835/1678] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:51:57.089843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:57.106932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5ccc5-281a-4737-93a8-590e92f90080 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:57.125672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:57.131913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.131913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.134951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] PATH_INFO: `/v3/projects/bc60f187260348f1a28df37b3ca5a973/users/9f906926289346ec9c3864db6f76eca5/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.151803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc60f187260348f1a28df37b3ca5a973, user_id=9f906926289346ec9c3864db6f76eca5, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:57.168816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9f84fc-bc58-4d01-88a6-d11bcfa0eef2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:57.264423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 844/1679] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:51:57 2022] PUT /identity/v3/projects/bc60f187260348f1a28df37b3ca5a973/users/9f906926289346ec9c3864db6f76eca5/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:57.271755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.272228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.272299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.401566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] MFA Rules not processed for user `9f906926289346ec9c3864db6f76eca5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:57.402933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:57.570446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd51a4-f471-4320-87d2-bef29eee60b9 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.571572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd51a4-f471-4320-87d2-bef29eee60b9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.572098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd51a4-f471-4320-87d2-bef29eee60b9 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.573294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:57.575167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 845/1680] 10.222.0.36 () {54 vars in 912 bytes} [Mon May 23 10:51:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:51:57.593713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acf590-868e-4f56-ad4b-e542627a71a1 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.593942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acf590-868e-4f56-ad4b-e542627a71a1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.594423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acf590-868e-4f56-ad4b-e542627a71a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.595634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 836/1681] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:57.608418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:57.612812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:57.667505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-11acf590-868e-4f56-ad4b-e542627a71a1 None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:57.670000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-11acf590-868e-4f56-ad4b-e542627a71a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:57.808056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88802f-f689-4468-ac3e-3b38757fa072 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:57.812876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:57.814892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.816028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.816028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.834544 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:57.835921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:57.839618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 846/1682] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:51:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:57.841291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:57.841817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:57.842321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:57.878035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 837/1683] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:57.896149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] MFA Rules not processed for user `7433cc7ed66f4e609332c56c936487e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:57.897747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.020283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:58.024074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.028085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 847/1684] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:58.034018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:58.035574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.063807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa9f0b-479d-4651-9df3-3acbaeca1ba5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:58.073700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:58.074501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.074852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.075193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.091131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:58.092739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf528bb4-aa49-41a3-95ed-65d23e82d562 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:58.114414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 848/1685] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:51:58.242911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97704a-3869-46e3-9a06-9d6698e7d79f service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:58.256509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:58.260447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.262409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.262993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] PATH_INFO: `/v3/projects/2707ec87254b4e55b636f5a5ac143858` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.266166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2707ec87254b4e55b636f5a5ac143858)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:58.270097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:58.497280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2707ec87254b4e55b636f5a5ac143858', 'request_id': 'req-23813004-c840-4495-9952-f782bccd1663'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:58.537874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2707ec87254b4e55b636f5a5ac143858', 'request_id': 'req-23813004-c840-4495-9952-f782bccd1663'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:58.562892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2707ec87254b4e55b636f5a5ac143858', 'request_id': 'req-23813004-c840-4495-9952-f782bccd1663'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:58.596883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] The token cache is being invalidate because project 2707ec87254b4e55b636f5a5ac143858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:51:58.597174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:58.602941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:58.608904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 838/1686] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:51:58 2022] DELETE /identity/v3/projects/2707ec87254b4e55b636f5a5ac143858 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:58.613154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674eb9d-df0f-44d5-b3a2-1bb53e578a92 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.613154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674eb9d-df0f-44d5-b3a2-1bb53e578a92 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.613154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674eb9d-df0f-44d5-b3a2-1bb53e578a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.615798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:58.616617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.658979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b674eb9d-df0f-44d5-b3a2-1bb53e578a92 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:58.660355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b674eb9d-df0f-44d5-b3a2-1bb53e578a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.708581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:58.745771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:51:58.851983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 849/1687] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:58.853652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.854116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.854501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.860473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23813004-c840-4495-9952-f782bccd1663 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:58.904238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:58.908258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.908553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.908839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] PATH_INFO: `/v3/users/448ee046d0bc462c8eeeb839632904f0` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.911744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:58.915216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:58.916931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=448ee046d0bc462c8eeeb839632904f0)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:58.930776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:58.944925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 850/1688] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:58.946034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:58.968085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951b0b3-4f23-42c4-bcfc-e9a193151075 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:58.980851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:58.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:58.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:58.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:58.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:51:58.984072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:58.985581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:59.018383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 851/1689] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:51:59.019485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:59.039670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111cc2f-4c8c-43b9-8263-31563c645d0a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:59.057093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:59.057667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:59.058102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:59.058409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:59.059194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:59.061240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:59.127125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448ee046d0bc462c8eeeb839632904f0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:59.129705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 852/1690] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:51:59 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:59.135029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:59.200627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448ee046d0bc462c8eeeb839632904f0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:59.202457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448ee046d0bc462c8eeeb839632904f0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:59.207473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448ee046d0bc462c8eeeb839632904f0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:59.234602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ddd3ca94-bdcc-4c12-acd6-5980d8acab26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448ee046d0bc462c8eeeb839632904f0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:51:59.284990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b3080-bc24-43d9-ac21-c5fdedb30e88 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:59.304828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:59.305626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:59.306001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:59.306349 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:59.307193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:59.309368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:59.328893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 839/1691] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:51:58 2022] DELETE /identity/v3/users/448ee046d0bc462c8eeeb839632904f0 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:51:59.342684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:59.342988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:59.352689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:59.390271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 853/1692] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:51:59 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:51:59.391489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:59.402499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:59.457736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb62e-ee62-49ee-96e5-ffc5e6f0b082 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:59.466133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:51:59.470481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:59.842798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:59.847447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:59.848491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:59.851744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:59.852817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:59.866193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 840/1693] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:51:59.869434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:51:59.875904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:59.877465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:51:59.903619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 854/1694] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:51:59.905748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:51:59.934669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010d1ae-ff56-4bfb-8ab2-61dce72111a8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:51:59.953619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:51:59.954711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:51:59.956854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:51:59.957309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:51:59.959111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:51:59.963950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:00.056561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 855/1695] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:00.059874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:00.079074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e4cb0-0074-49f0-9463-bb722f014c74 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:00.089429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:00.090630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:00.091064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:00.091514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] PATH_INFO: `/v3/projects/e387b9f4a9de4c9c89da58207ac3b76e/users/2773a357a9ce4cf191a0a2df7bfad7a1/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:00.095008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:00.148013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e387b9f4a9de4c9c89da58207ac3b76e, user_id=2773a357a9ce4cf191a0a2df7bfad7a1, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:00.162410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:00.187606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:00.212028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 856/1696] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:00 2022] PUT /identity/v3/projects/e387b9f4a9de4c9c89da58207ac3b76e/users/2773a357a9ce4cf191a0a2df7bfad7a1/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:00.212627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2005159-cb94-4949-ae89-754f1395fd8d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:00.214696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:00.218934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:00.455176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:00.455858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:00.456141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:00.456836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:00.495805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a76cc-d56b-4f8a-95d6-793d8ba58b57 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:00.525406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:00.525406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d128b5-312e-4ef4-bfcb-e471e2d812b7 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:00.531785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:00.534613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:00.535049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:00.535825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] PATH_INFO: `/v3/users/297487ae0a294490b56a967ce665e63b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:00.539511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=297487ae0a294490b56a967ce665e63b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:00.541940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 841/1697] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:00.543805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:00.545460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:00.545760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:00.548684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:00.642231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297487ae0a294490b56a967ce665e63b'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:00.660520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297487ae0a294490b56a967ce665e63b'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:00.661376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297487ae0a294490b56a967ce665e63b'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:00.662062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297487ae0a294490b56a967ce665e63b'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:00.672909 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b500b2ca-d3e6-4246-9710-d38767b42c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297487ae0a294490b56a967ce665e63b'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:00.718371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] MFA Rules not processed for user `2773a357a9ce4cf191a0a2df7bfad7a1`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:00.719781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:00.727632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 857/1698] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:00 2022] DELETE /identity/v3/users/297487ae0a294490b56a967ce665e63b => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:00.733863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:00.735895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:00.736304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:00.784800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:00.789123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:01.027571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 858/1699] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:01.040864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 842/1700] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:01.045454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.046953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:01.070589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:01.083970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2afaf-eb2a-48b7-8ba9-8bf12c8d9dd9 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:01.096960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.098906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:01.129131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:01.132083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.132413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.132658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.133376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:01.135554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:01.147114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f93686-6c21-4f3a-97fa-e61ff223f103 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:01.230580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 859/1701] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:01 2022] POST /identity/v3/projects => generated 367 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:01.234845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.408666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:01.413443 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.414260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.414743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.463564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:01.466683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612654c7-6873-4fe7-b0c9-ecda7f806cbd service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:01.476581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913cd4-db4d-46e3-b0bb-23d53b84b375 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:01.489358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:01.493429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.493789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.495045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.495860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:01.501423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:01.518640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 843/1702] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:01.520642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc28e8-f581-4a8b-b33e-b54b32e4e7d7 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.528080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc28e8-f581-4a8b-b33e-b54b32e4e7d7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.528666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc28e8-f581-4a8b-b33e-b54b32e4e7d7 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.532678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 844/1703] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:52:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:52:01.575957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc28e8-f581-4a8b-b33e-b54b32e4e7d7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.576183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 860/1704] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:01 2022] POST /identity/v3/users => generated 290 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:01.625807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc28e8-f581-4a8b-b33e-b54b32e4e7d7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:01.656732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:01.657818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.658317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.658850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.659834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:01.662921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67032dee-81fe-4168-9dd8-5200642b8637 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:01.719652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.721441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 845/1705] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:01.723287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb05da-bed3-4e5b-9c5a-315e7b677e72 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.725260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb05da-bed3-4e5b-9c5a-315e7b677e72 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.725770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb05da-bed3-4e5b-9c5a-315e7b677e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.744782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f721b-c919-4741-b9cc-849e3f07d563 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:01.776696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:01.778899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:01.796824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:01.798110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] PATH_INFO: `/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6/users/a9281914ce084652b41e9f71e3a1a5a0/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:01.799501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b9cb05da-bed3-4e5b-9c5a-315e7b677e72 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:01.812614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b9cb05da-bed3-4e5b-9c5a-315e7b677e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:01.853174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdbe4b5a48a43cf85e8988b193e53f6, user_id=a9281914ce084652b41e9f71e3a1a5a0, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:01.868400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:01.917226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 861/1706] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:01 2022] PUT /identity/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6/users/a9281914ce084652b41e9f71e3a1a5a0/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:01.919814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:01.932766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:02.140566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 846/1707] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:52:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:02.159188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e39cb-4e2a-4c2c-8ea2-d2a51c4b6d9b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:02.159696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e39cb-4e2a-4c2c-8ea2-d2a51c4b6d9b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:02.162345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e39cb-4e2a-4c2c-8ea2-d2a51c4b6d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:02.171580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775a2b3-feab-4d3c-84b5-2e544a86b7ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:02.189041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:02.195999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:02.196337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:02.196631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:02.197501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:02.205380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:02.245514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bf2e39cb-4e2a-4c2c-8ea2-d2a51c4b6d9b None None] MFA Rules not processed for user `8e98c8ceac6b47079423b45685c5ba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:02.249186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bf2e39cb-4e2a-4c2c-8ea2-d2a51c4b6d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:02.257098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 862/1708] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:02.270591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:02.331209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb182-7eec-4f01-82cc-b488e92f0661 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:02.349247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:02.352990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:02.355546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:02.357632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] PATH_INFO: `/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6/users/a9281914ce084652b41e9f71e3a1a5a0/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:02.363164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdbe4b5a48a43cf85e8988b193e53f6, user_id=a9281914ce084652b41e9f71e3a1a5a0, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:02.373159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:02.418785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 863/1709] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:02 2022] PUT /identity/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6/users/a9281914ce084652b41e9f71e3a1a5a0/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:02.418785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:02.421875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:02.472381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 847/1710] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:02.473644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:02.477336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:02.477336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:02.572435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] MFA Rules not processed for user `a9281914ce084652b41e9f71e3a1a5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:02.579610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:02.622101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:02.816000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758d039-d412-430b-a1cb-f932b180cea8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:02.838973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:02.840037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:02.840333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:02.840617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] PATH_INFO: `/v3/projects/9d2278c536e14f7e96a3b01a18cd19ad` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:02.844521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2278c536e14f7e96a3b01a18cd19ad)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:02.845595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: WARNING py.warnings [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 10:52:02.845595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: warnings.warn(msg) May 23 10:52:02.845595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: May 23 10:52:02.847189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 848/1711] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:02.853006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:02.859519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:02.867915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.045815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2278c536e14f7e96a3b01a18cd19ad', 'request_id': 'req-672003d6-49fd-40aa-bba2-a123c6d75694'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:03.062324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2278c536e14f7e96a3b01a18cd19ad', 'request_id': 'req-672003d6-49fd-40aa-bba2-a123c6d75694'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:03.090121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:03.092005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2278c536e14f7e96a3b01a18cd19ad', 'request_id': 'req-672003d6-49fd-40aa-bba2-a123c6d75694'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:03.123417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] The token cache is being invalidate because project 9d2278c536e14f7e96a3b01a18cd19ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:03.123931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:03.125452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:03.133755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 864/1712] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:02 2022] DELETE /identity/v3/projects/9d2278c536e14f7e96a3b01a18cd19ad => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:03.133755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:03.135199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.174703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28412baf-ca65-455e-acdf-6626fcf5f754 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:03.371432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672003d6-49fd-40aa-bba2-a123c6d75694 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:03.422047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:03.427582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:03.430288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:03.432172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:03.438924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.446184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:03.448750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:03.449067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:03.449349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:03.453403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.504061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:03.506100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ced6b-8a04-4828-b955-f6565b00413c service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:03.546418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 849/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:03.550703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:03.551197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:03.551899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:03.591975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:03.593618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.653670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:03.655179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:03.686729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 865/1714] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:03.687368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:03.690899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.722810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:03.806714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 850/1715] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:03.807916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:03.817150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.850558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7242e-c592-466a-838f-acdbadf9bb2f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:03.868058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:03.868728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:03.869018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:03.869300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:03.871385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:03.928287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfdb3-8e58-44eb-880c-ab07b95b1870 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:03.942729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:03.943883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:03.944341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:03.944823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] PATH_INFO: `/v3/projects/d8013fb326a54588b6fc29be5476b99d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:03.953238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8013fb326a54588b6fc29be5476b99d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:03.963916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:03.977867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:03.986161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5285d-c0c5-41dd-964d-ff4e8e843551 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:03.997042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 851/1716] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:04.010237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.010745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.012792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.099943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:04.108387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:04.124359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 852/1717] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:04.128878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.144422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:04.155858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b97a6-0ccb-4ad9-a82e-eb1110b79070 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:04.166676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8013fb326a54588b6fc29be5476b99d', 'request_id': 'req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.190211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8013fb326a54588b6fc29be5476b99d', 'request_id': 'req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.210500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8013fb326a54588b6fc29be5476b99d', 'request_id': 'req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.247520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] The token cache is being invalidate because project d8013fb326a54588b6fc29be5476b99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:04.247815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.254973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.261145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 866/1718] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:03 2022] DELETE /identity/v3/projects/d8013fb326a54588b6fc29be5476b99d => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:04.264971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.277811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:04.300012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:04.301348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.302149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.302646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.303593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:04.304103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:04.306009 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:04.336557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 853/1719] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:52:04.340331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.344413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:04.383327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a620af-495d-471e-a48d-310e28b6a55f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:04.396779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:04.397994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.398560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.399067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.400045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:04.405808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:04.468557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 854/1720] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:04 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:04.472253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.479731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f81b1-233e-4d72-ab2d-d3e1c8a905ac admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:04.602175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:04.658675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:04.659948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.660066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.667482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] PATH_INFO: `/v3/users/9f906926289346ec9c3864db6f76eca5` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.676149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f906926289346ec9c3864db6f76eca5)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:04.680468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:04.683789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b0e28-e4c8-4c74-9b29-a8dbf413e44f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:04.709885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:04.711028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.712680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.713107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.715285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:04.721894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:04.818856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 855/1721] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:04 2022] POST /identity/v3/users => generated 285 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:04.821140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.870035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22f6ee-2968-49d3-a9c0-fcf3a6f9c30e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:04.886491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f906926289346ec9c3864db6f76eca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.888594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:04.890416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:04.891667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:04.892379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:04.895391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:04.898264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:04.923293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f906926289346ec9c3864db6f76eca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.924157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f906926289346ec9c3864db6f76eca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.924836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f906926289346ec9c3864db6f76eca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.953340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f906926289346ec9c3864db6f76eca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:04.971986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 856/1722] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:04.980027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:04.984088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:05.020429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a6ff4-dabd-465c-a38c-b72a78e65d7b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:05.071196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:05.073042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:05.073348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:05.074304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:05.075753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:05.082625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48732ee2-3825-4833-986f-90418d3a6327 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:05.089002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 867/1723] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:04 2022] DELETE /identity/v3/users/9f906926289346ec9c3864db6f76eca5 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:05.116769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 857/1724] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:05.129443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:05.294333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52d2f3-5451-45e4-8b19-9e3940fdfb35 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:05.307796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:05.308993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:05.309424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:05.309843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] PATH_INFO: `/v3/projects/93f3eba472344871a2e6d553a0effd10/users/75c2539778f3436091611b7f9d0ae251/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:05.345721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93f3eba472344871a2e6d553a0effd10, user_id=75c2539778f3436091611b7f9d0ae251, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:05.350044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:05.391529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:05.392064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:05.392064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:05.392676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 868/1725] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:05 2022] PUT /identity/v3/projects/93f3eba472344871a2e6d553a0effd10/users/75c2539778f3436091611b7f9d0ae251/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:05.475681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] MFA Rules not processed for user `75c2539778f3436091611b7f9d0ae251`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:05.477097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:05.591210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:05.729153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 858/1726] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:05.745600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:05.916808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:06.014061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6993aeaa-d7d6-4f3b-bd98-0fa3f002bd1f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:06.025622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:06.026990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:06.031100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:06.032064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:06.033353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:06.039055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:06.161310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 869/1727] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:05 2022] POST /identity/v3/projects => generated 369 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:06.165176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:06.191895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553500d-680a-49cc-8923-1ae61364ed72 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:06.430602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:06.431483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:06.436719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:06.437056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:06.441139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:06.452235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:06.528143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 859/1728] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:06 2022] POST /identity/v3/users => generated 291 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:06.535243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:06.557160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c55-d997-43bc-826d-4055f91780ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:06.585783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:06.586792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:06.587087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:06.587368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:06.588106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:06.589752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:06.634956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:06.637670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 870/1729] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:06.654578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87379f95-a933-4fe7-8013-b3cadee265ab admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:06.671810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:06.673306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:06.674045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:06.674644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] PATH_INFO: `/v3/projects/9ff845b3ca5b4c838210447496aa0911/users/9291e755af1944eeb35dc1d4bbddcd44/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:06.727559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff845b3ca5b4c838210447496aa0911, user_id=9291e755af1944eeb35dc1d4bbddcd44, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:06.735347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:06.794316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 860/1730] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:06 2022] PUT /identity/v3/projects/9ff845b3ca5b4c838210447496aa0911/users/9291e755af1944eeb35dc1d4bbddcd44/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:06.797072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:06.815787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a8c64-da84-4ab0-a65a-836ca9dfe1a1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.083218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.083947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.084252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.084530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.085269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.092537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5429251-cbd0-4fe1-8bbe-2457e98a4c5e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:07.111359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 871/1731] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:07.119787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:07.135933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691ed6e-6fb2-4d97-ba9a-1cd61286e9d7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.141897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.143386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.143768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.144139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] PATH_INFO: `/v3/projects/9ff845b3ca5b4c838210447496aa0911/users/9291e755af1944eeb35dc1d4bbddcd44/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.146752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff845b3ca5b4c838210447496aa0911, user_id=9291e755af1944eeb35dc1d4bbddcd44, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.150427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:07.178731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 861/1732] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:07 2022] PUT /identity/v3/projects/9ff845b3ca5b4c838210447496aa0911/users/9291e755af1944eeb35dc1d4bbddcd44/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:07.186824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.189141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.189579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.252447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] MFA Rules not processed for user `9291e755af1944eeb35dc1d4bbddcd44`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:07.253814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:07.356605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:07.359016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:07.367604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 872/1733] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:07.379547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:07.381011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:07.433141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:07.444969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dfff21-5935-4a2c-b1cd-6ccdded8aae0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.450509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.451399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.451762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.452130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.476876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.478456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80d024-d681-4693-9eec-9fdbad9a37ab service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:07.478944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0f422-a14f-4298-a34e-9d6c266a2324 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.485654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.486642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.487084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.487581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] PATH_INFO: `/v3/projects/bc60f187260348f1a28df37b3ca5a973` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.488904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc60f187260348f1a28df37b3ca5a973)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.492131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 873/1734] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:07.502458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:07.556964 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:07.587446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc60f187260348f1a28df37b3ca5a973', 'request_id': 'req-67d8ee52-e0f6-4987-af28-c94eddee1cfa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:07.609558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc60f187260348f1a28df37b3ca5a973', 'request_id': 'req-67d8ee52-e0f6-4987-af28-c94eddee1cfa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:07.620852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc60f187260348f1a28df37b3ca5a973', 'request_id': 'req-67d8ee52-e0f6-4987-af28-c94eddee1cfa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:07.647727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] The token cache is being invalidate because project bc60f187260348f1a28df37b3ca5a973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:07.648030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:07.648757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67d8ee52-e0f6-4987-af28-c94eddee1cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:07.651171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 862/1735] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:07 2022] DELETE /identity/v3/projects/bc60f187260348f1a28df37b3ca5a973 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:07.656060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.656377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.656674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.688180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:07.689542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:07.813608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 874/1736] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:07.822764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.823375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.823830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.862235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:07.863617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:07.878418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 863/1737] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:07.882811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:07.898443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c37e0-4a78-4a59-98e0-e854c3d76e02 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.905451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.906444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.906851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.907131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.907876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:07.908187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.910485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:07.935704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 875/1738] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:07.936206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:07.956056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d723fd3b-f78b-41e9-af4d-f5de1c4511d2 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:07.971172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:07.972291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:07.972644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:07.973244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:07.974051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:07.976301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.013368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 864/1739] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:07 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:08.014971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:08.030326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fd3b9-e28a-4d0c-94fe-37ba17025eaf admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:08.144351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:08.145316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.145637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.149499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.150592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:08.154062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.191122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:08.203320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 876/1740] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:08 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:08.211101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00404b-6e23-4224-9640-9858b7f48e54 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:08.251144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:08.252297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.252736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.253268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.254220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:08.256202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.299411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:08.300013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 865/1741] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:08.311746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3fdf-8e92-40a3-9f4a-aa87b14fe6b3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:08.323746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:08.324526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.324818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.325101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.325827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:08.350360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25263bb4-b310-4bc2-ad42-21335cadde84 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.377663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 877/1742] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:08.377791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:08.392457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c976-9b6c-4909-9f99-9a269250a8c2 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:08.407627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:08.409047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.409482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.409900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] PATH_INFO: `/v3/projects/3b495d67b32a4f6da27fe0be6650d2fd/users/ca8e7d3ea52f4a3d942b687633d69ddf/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.446984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b495d67b32a4f6da27fe0be6650d2fd, user_id=ca8e7d3ea52f4a3d942b687633d69ddf, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:08.449979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.473079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 866/1743] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:08 2022] PUT /identity/v3/projects/3b495d67b32a4f6da27fe0be6650d2fd/users/ca8e7d3ea52f4a3d942b687633d69ddf/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:08.474857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.475155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.475809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.535641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:08.544130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:08.685253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:08.715081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 878/1744] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:08.726853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:08.728763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:08.804656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af541b80-0515-470c-893a-b950479d9b5d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:08.808641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:08.809378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:08.809732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:08.810136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:08.821008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:08.822705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6265bf62-e64a-4f42-8f8b-0358cffe3d5f service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:08.846154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 867/1745] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:09.314730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:09.316733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:09.340362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8908bb-3a6b-4924-ba53-fc152490e4e3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:09.356227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:09.357416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:09.358110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:09.358547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] PATH_INFO: `/v3/users/2773a357a9ce4cf191a0a2df7bfad7a1` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:09.360324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2773a357a9ce4cf191a0a2df7bfad7a1)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:09.362624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:09.476418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2773a357a9ce4cf191a0a2df7bfad7a1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:09.488671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2773a357a9ce4cf191a0a2df7bfad7a1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:09.497091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2773a357a9ce4cf191a0a2df7bfad7a1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:09.500751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2773a357a9ce4cf191a0a2df7bfad7a1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:09.516180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2773a357a9ce4cf191a0a2df7bfad7a1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:09.554314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 879/1746] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:09 2022] DELETE /identity/v3/users/2773a357a9ce4cf191a0a2df7bfad7a1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:09.558837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:09.559378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:09.559862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:09.581869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:09.584038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:09.654216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:09.669707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 868/1747] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:09.681009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:09.682917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:09.759714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b10294-252f-43e7-a4da-2f59cd1dbc1a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:09.771694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:09.773029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:09.774798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:09.777634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:09.800356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:09.804497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56e862-559d-49ba-8825-e8adb8bc3c2f service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:09.826173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 880/1748] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:10.198501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:10.201738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:10.242035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853cd92-9983-4f2e-9619-df619eb4b8ef None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:10.252918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:10.254031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:10.254431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:10.254809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] PATH_INFO: `/v3/users/75c2539778f3436091611b7f9d0ae251` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:10.256148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c2539778f3436091611b7f9d0ae251)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:10.258529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:10.332013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c2539778f3436091611b7f9d0ae251'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.347675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c2539778f3436091611b7f9d0ae251'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.348415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c2539778f3436091611b7f9d0ae251'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.349203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c2539778f3436091611b7f9d0ae251'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.358435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c2539778f3436091611b7f9d0ae251'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.377980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:10.378468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:10.378546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:10.381177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 869/1749] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:10 2022] DELETE /identity/v3/users/75c2539778f3436091611b7f9d0ae251 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:10.401302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:10.402896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:10.440066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:10.445143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:10.483369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 881/1750] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:10.495582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:10.497068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:10.511089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:10.533806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27023451-24e4-4845-8e68-875c0f94075e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:10.540748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:10.542029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:10.542463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:10.542959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] PATH_INFO: `/v3/projects/e387b9f4a9de4c9c89da58207ac3b76e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:10.544401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e387b9f4a9de4c9c89da58207ac3b76e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:10.547201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:10.574711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d36c4-b6c8-4011-ac48-995558bf02f8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:10.580700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:10.581891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:10.582605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:10.584211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:10.614482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e387b9f4a9de4c9c89da58207ac3b76e', 'request_id': 'req-68c928c0-2084-4aa8-9b36-1162e86370a2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.615122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:10.617746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:10.624181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e387b9f4a9de4c9c89da58207ac3b76e', 'request_id': 'req-68c928c0-2084-4aa8-9b36-1162e86370a2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.633690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 882/1751] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:10.636007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e387b9f4a9de4c9c89da58207ac3b76e', 'request_id': 'req-68c928c0-2084-4aa8-9b36-1162e86370a2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.650335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:10.656775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] The token cache is being invalidate because project e387b9f4a9de4c9c89da58207ac3b76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:10.657061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.657844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-68c928c0-2084-4aa8-9b36-1162e86370a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.660533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 870/1752] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:10 2022] DELETE /identity/v3/projects/e387b9f4a9de4c9c89da58207ac3b76e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:10.664593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:10.674932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a14d1-9547-4d31-9987-0e3a01b60b5b service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:10.746375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:10.747303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:10.747560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:10.748111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] PATH_INFO: `/v3/users/a9281914ce084652b41e9f71e3a1a5a0` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:10.749647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9281914ce084652b41e9f71e3a1a5a0)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:10.752613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:10.819426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9281914ce084652b41e9f71e3a1a5a0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.829859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9281914ce084652b41e9f71e3a1a5a0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.830917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9281914ce084652b41e9f71e3a1a5a0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.831552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9281914ce084652b41e9f71e3a1a5a0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.852376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9281914ce084652b41e9f71e3a1a5a0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:10.888103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 883/1753] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:10 2022] DELETE /identity/v3/users/a9281914ce084652b41e9f71e3a1a5a0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:11.221001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.225211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.226066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.277039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:11.279239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:11.310710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] MFA Rules not processed for user `ca8e7d3ea52f4a3d942b687633d69ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:11.312172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:11.337879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:11.383501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2e53-5ef1-451a-b406-e3fb826b36f2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:11.401758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:11.406079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.406433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.407932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] PATH_INFO: `/v3/projects/93f3eba472344871a2e6d553a0effd10` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.410685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=93f3eba472344871a2e6d553a0effd10)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:11.416869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 871/1754] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:11.417016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:11.425144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:11.427148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:11.462232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09181045-fad7-4c14-8e27-da117fd5c6f8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:11.468408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:11.469082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.469522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.469977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.470845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:11.472947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:11.485577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93f3eba472344871a2e6d553a0effd10', 'request_id': 'req-43d9203e-21cb-4404-bd0d-398daca107ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:11.498340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93f3eba472344871a2e6d553a0effd10', 'request_id': 'req-43d9203e-21cb-4404-bd0d-398daca107ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:11.508559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93f3eba472344871a2e6d553a0effd10', 'request_id': 'req-43d9203e-21cb-4404-bd0d-398daca107ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:11.513099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 872/1755] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:11 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:11.516995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:11.538025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] The token cache is being invalidate because project 93f3eba472344871a2e6d553a0effd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:11.538646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:11.543838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:11.546137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 884/1756] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:11 2022] DELETE /identity/v3/projects/93f3eba472344871a2e6d553a0effd10 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:11.550014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:11.552293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:11.633415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec9266-a334-4d49-86a3-143cb66c6392 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:11.646106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:11.648884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.650333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.650793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.651642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:11.658826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce772499-bf44-4200-bbde-db172558ef3e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:11.679195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9203e-21cb-4404-bd0d-398daca107ca admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:11.688403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:11.689318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.689606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.689887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] PATH_INFO: `/v3/users/9291e755af1944eeb35dc1d4bbddcd44` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.691384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9291e755af1944eeb35dc1d4bbddcd44)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:11.694128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:11.724255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 873/1757] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:11 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:11.725666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.725974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.726260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.734866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 874/1758] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:52:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:52:11.735754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:11.737481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:11.805051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72447b-a493-498c-8e05-3b19c6d94e65 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:11.835184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:11.835875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.836233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.836424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.837165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:11.838860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b3e38-c460-42ee-b0fc-33e87020f2bc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:11.882416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 875/1759] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:11.886974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:11.888215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:11.888844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:11.938518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:11.939766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.013903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:12.014929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9291e755af1944eeb35dc1d4bbddcd44'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.026249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9291e755af1944eeb35dc1d4bbddcd44'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.027206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9291e755af1944eeb35dc1d4bbddcd44'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.027978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9291e755af1944eeb35dc1d4bbddcd44'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.041019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9291e755af1944eeb35dc1d4bbddcd44'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.043019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:12.061224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 876/1760] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:52:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:12.061965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.063284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.073299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 885/1761] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:11 2022] DELETE /identity/v3/users/9291e755af1944eeb35dc1d4bbddcd44 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:12.074471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.076386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.084761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abef25-6a72-445a-97ad-dcda9c90dc6f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.162671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.163682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.163994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.164264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] PATH_INFO: `/v3/projects/45fad236c8b04df6889bb6321643a848/users/ffb172ce90f946b89eea0fac6e7c16fe/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.168586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54a3cd-904c-4948-852e-821d966a10f3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.174815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.176067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.176523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.177280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] PATH_INFO: `/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.179087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdbe4b5a48a43cf85e8988b193e53f6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.179809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fad236c8b04df6889bb6321643a848, user_id=ffb172ce90f946b89eea0fac6e7c16fe, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.181399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.182668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.200378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 877/1762] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:12 2022] PUT /identity/v3/projects/45fad236c8b04df6889bb6321643a848/users/ffb172ce90f946b89eea0fac6e7c16fe/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:12.203729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.205501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.258443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdbe4b5a48a43cf85e8988b193e53f6', 'request_id': 'req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.284344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdbe4b5a48a43cf85e8988b193e53f6', 'request_id': 'req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.295319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdbe4b5a48a43cf85e8988b193e53f6', 'request_id': 'req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.310187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] The token cache is being invalidate because project dbdbe4b5a48a43cf85e8988b193e53f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:12.310378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.310951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.313569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 886/1763] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:12 2022] DELETE /identity/v3/projects/dbdbe4b5a48a43cf85e8988b193e53f6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:12.314485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.315836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.337847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d69af-f4e7-4d4c-861d-7886172aff8c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.426315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:12.481648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.482382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.482668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.482951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.485774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08c3a7-6c31-4073-b91f-2c138ca79c3f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.487878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.507775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.508534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.508821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.509102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.509835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.512087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d397b0-db69-4842-8d90-ffb6e3f89617 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.521807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.523467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.536890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 887/1764] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:12.538398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.538691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.538972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.550863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 878/1765] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:12.553800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.564664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:12.575428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be846f9-a5b7-4a93-9154-961574782f1a service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.577135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:12.580484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.586497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.587409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.587694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.587970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] PATH_INFO: `/v3/projects/45fad236c8b04df6889bb6321643a848/users/ffb172ce90f946b89eea0fac6e7c16fe/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.591688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fad236c8b04df6889bb6321643a848, user_id=ffb172ce90f946b89eea0fac6e7c16fe, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.596153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a2e55-82ff-480b-968f-32a0b2d4b6ee admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.601097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 888/1766] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:12.601989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.605426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.624914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 879/1767] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:12 2022] PUT /identity/v3/projects/45fad236c8b04df6889bb6321643a848/users/ffb172ce90f946b89eea0fac6e7c16fe/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:12.627022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.627312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.627590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.631584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b7afe-19ab-49c9-be30-b878d1e28e27 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.673021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:12.677563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.752337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.753446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.753926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.754233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] PATH_INFO: `/v3/projects/9ff845b3ca5b4c838210447496aa0911` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.755999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff845b3ca5b4c838210447496aa0911)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.758617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.780102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 880/1768] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:12.784064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.799636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cad597-6a33-4b47-8c7a-18076b96ef68 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.805740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.806499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.806790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.807070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.807967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:12.808328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.810423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.829629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff845b3ca5b4c838210447496aa0911', 'request_id': 'req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.834514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 881/1769] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:52:12.838214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff845b3ca5b4c838210447496aa0911', 'request_id': 'req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.838934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:12.849104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff845b3ca5b4c838210447496aa0911', 'request_id': 'req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.852505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58615c-704c-4d9e-a006-9d0303536c85 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:12.867026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] The token cache is being invalidate because project 9ff845b3ca5b4c838210447496aa0911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:12.868305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.869302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0da9f745-4ddc-445b-a87f-3cf6ba1b4336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:12.877674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 889/1770] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:12 2022] DELETE /identity/v3/projects/9ff845b3ca5b4c838210447496aa0911 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:12.880632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0e970-3426-4084-afc0-b03009b722ea None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.881167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0e970-3426-4084-afc0-b03009b722ea None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.881600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0e970-3426-4084-afc0-b03009b722ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.907575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3db0e970-3426-4084-afc0-b03009b722ea None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:12.909250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3db0e970-3426-4084-afc0-b03009b722ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:12.946576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:12.947248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:12.947531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:12.947803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:12.948510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:12.953711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:12.995905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 882/1771] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:12 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:13.001015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.004142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:13.015594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 890/1772] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:13.015594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.015594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.015594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.048728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:13.051196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:13.082470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:13.091899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:13.116190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805b0ad-a8dc-4467-9484-92230d4b9034 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.123455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.124416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.124713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.125104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.126479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.130417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.152360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 891/1773] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:13.156968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.157514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 883/1774] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:13.161892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.170452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644fbb6-2201-4950-8e8e-8a4a90ab7423 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.175436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7098e-c827-4b19-8543-6991ba8999a9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.177238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.178156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.178454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.179323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.180145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:13.180469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.182751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.183507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.184396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.184748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.185097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.186472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.188176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.201800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 884/1775] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:13.205991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.211188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 892/1776] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:13.212404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.218330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340c265-7fa7-4007-b8b9-f6804679438a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.223477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca3616-cde6-429c-acd8-e20ffc72672b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.224774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.225572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.225923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.226383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.227120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.228790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.229304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.230051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.245408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 885/1777] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:13.246787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.248589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.248955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.249706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.252872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.261479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83217f79-7178-43b9-ba2e-61aee1949865 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.268458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.269628 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.270367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.270652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] PATH_INFO: `/v3/projects/4f52bfa5c8304ec3b5249a6dcceac5ac/users/c1b6a2d2249b4c6489995e5206e9e420/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.293975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 893/1778] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:13.298461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.299718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f52bfa5c8304ec3b5249a6dcceac5ac, user_id=c1b6a2d2249b4c6489995e5206e9e420, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.303577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11bfc6-3547-4e57-85b5-3e857db91948 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.327547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 886/1779] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:13 2022] PUT /identity/v3/projects/4f52bfa5c8304ec3b5249a6dcceac5ac/users/c1b6a2d2249b4c6489995e5206e9e420/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:13.337892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.338498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.338570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.396233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] MFA Rules not processed for user `c1b6a2d2249b4c6489995e5206e9e420`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:13.397972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:13.440726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda0221-82bd-4c37-9c7c-3c2ef7e77751 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.497072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 887/1780] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:13.498043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.558133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.558879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.559167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.559441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.560251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.562440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.589542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 894/1781] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:13.593179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.605247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df398c1d-c271-4fa7-8962-49e2338c4d2b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.606839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c399a-75ab-4af5-aab3-aa16e11f48fb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.611472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.612395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.612880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.613314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.614805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.617527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.617854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.618696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.619219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.619747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.620957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.624466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.640338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 895/1782] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:13.643420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:13.646181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.657801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7974089-bc00-454c-ac8a-725c5e056ff2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.659129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 888/1783] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:13.660024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.678911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:13.725572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.726406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.726767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.727126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.728041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.729802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.741118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 896/1784] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:13.745262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.747826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6824ab-8902-4445-8c00-3c506176447e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.755223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.755921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.756210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.756552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.757228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd2874-8dea-4ed8-ac79-0ea0ae127df7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.757809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.760922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.763805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.764815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.765193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.765551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] PATH_INFO: `/v3/projects/b4e0d7c21dc9483383fae87c5990262f/users/f0985089eee647acb6b5396d88e12f12/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.791200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 889/1785] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:13.796762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e0d7c21dc9483383fae87c5990262f, user_id=f0985089eee647acb6b5396d88e12f12, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.799101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.801495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b229d8-e407-46b0-abdb-240143bcf2b3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.814864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44366e95-dad7-455d-b1d3-50f47d08eff4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.822605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.823553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.824030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.824447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.825364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.825696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 897/1786] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:13 2022] PUT /identity/v3/projects/b4e0d7c21dc9483383fae87c5990262f/users/f0985089eee647acb6b5396d88e12f12/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:13.828153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.836383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.836697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.837015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.850988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 890/1787] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:13.853219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.865164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dec50-4142-4fa4-8b4a-2c8cc01278dc admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.870576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] MFA Rules not processed for user `f0985089eee647acb6b5396d88e12f12`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:13.872674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:13.939249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.940007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.940674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.940674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.941277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:13.942973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296af4-6128-473f-a08f-4bdebc558daf admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:13.953204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 891/1788] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:13.954521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7df6-aa0b-4e1b-b7bf-bfe04fc9b309 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.954789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7df6-aa0b-4e1b-b7bf-bfe04fc9b309 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.955069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f7df6-aa0b-4e1b-b7bf-bfe04fc9b309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:13.976124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 898/1789] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:13.980949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:13.989481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0d2f7df6-aa0b-4e1b-b7bf-bfe04fc9b309 None None] MFA Rules not processed for user `ffb172ce90f946b89eea0fac6e7c16fe`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:13.990864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0d2f7df6-aa0b-4e1b-b7bf-bfe04fc9b309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:13.991675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee157b-7423-46e7-ab7a-71210056073b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:13.996393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:13.997294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:13.997634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:13.997940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] PATH_INFO: `/v3/projects/6aaf99766b494f0e8c3708f5fb73967a/users/46aa8039b0f74e7a870cc14a13ff4bad/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.013939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf99766b494f0e8c3708f5fb73967a, user_id=46aa8039b0f74e7a870cc14a13ff4bad, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.018415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.039138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 899/1790] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:13 2022] PUT /identity/v3/projects/6aaf99766b494f0e8c3708f5fb73967a/users/46aa8039b0f74e7a870cc14a13ff4bad/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:14.039733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.089988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:14.099647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 892/1791] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:14.104204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.105196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.106139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.128275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02656339-bf48-4c7c-8ee0-4526d4ac7d04 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.137050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.137730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.138203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.138766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.139227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.141262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.147660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:14.149521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] MFA Rules not processed for user `46aa8039b0f74e7a870cc14a13ff4bad`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:14.152368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:14.192987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 900/1792] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:14.192987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.198773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:14.253251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 893/1793] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:14.257320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.267124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47236b-a4c0-4b88-a8b1-97677b685318 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.271327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a546c5-68d2-4200-936d-d0dd32cb3f59 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.275218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.275920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.276206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.276488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.278446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:14.344677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.345562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.345856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.346181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.347586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.349678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.350045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.351314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.360989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 901/1794] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:14.361698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.374222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e4ac2-75c1-4ffb-87ad-7a98985d0733 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.378081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 894/1795] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:14.379302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.383409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.383409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.383409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.383409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.384421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.385078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.401154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5be111-91e4-46e5-a282-856aef4b1f8c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.408477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.408880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.409175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.409811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.410758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.413145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.420102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 902/1796] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:14.421492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.425682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 895/1797] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:14.431074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.444047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9039-f16e-45d3-a274-846bc5e85fde admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.517916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.518853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.518967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.519171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.520523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.524502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.543850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 896/1798] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:14.550347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc98e3-65f4-4104-be37-2d3d40b8b3fd admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.552192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.566781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:14.569059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.569762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.570151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.570437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.571471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.574496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.575688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a914ea-5624-4595-abfc-caf45cdab872 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.583836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.585289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.586061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.586423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] PATH_INFO: `/v3/projects/6519eb653f68444286b7b3174ea0cea5/users/b7eb778a4352490590e4945c8beb92ec/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.599932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 903/1799] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:14.604655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.607266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6519eb653f68444286b7b3174ea0cea5, user_id=b7eb778a4352490590e4945c8beb92ec, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.610027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689838f4-e0d2-4c47-94cc-c7c17bcb0ce8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.619819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1b768-429b-4998-8683-21ac780b9155 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.626216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.627845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.628208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.628544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.630357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.632276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.634374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:14.640643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 897/1800] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:14 2022] PUT /identity/v3/projects/6519eb653f68444286b7b3174ea0cea5/users/b7eb778a4352490590e4945c8beb92ec/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:14.655244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.655244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.655244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.656483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 904/1801] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:14.657453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.669581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ed8f9-ad30-4beb-878c-e47efda3fea7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.691354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] MFA Rules not processed for user `b7eb778a4352490590e4945c8beb92ec`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:14.693724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:14.776304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.777684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.777684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.778144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] PATH_INFO: `/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2/users/e5d251fc19f74731aa9ee99af8a7b3ef/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.797209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf063a6ffc6e4ab08f3bbe3919becbd2, user_id=e5d251fc19f74731aa9ee99af8a7b3ef, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.802927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.824286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 898/1802] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:14.825613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.832382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 905/1803] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:14 2022] PUT /identity/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2/users/e5d251fc19f74731aa9ee99af8a7b3ef/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:14.837033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.840554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a667982-2b2f-4bfd-95fc-36fbb739eb67 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.901325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.902017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.902300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.902604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.903319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.903696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb7ccb-81a5-44e0-b054-11485f131c57 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.905265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.908371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.909039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.909325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.909605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.910364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.912479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:14.922744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 906/1804] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:14.927407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.934822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 899/1805] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:14.935733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:14.937428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0d06d-2807-43df-8eab-b78e91933c65 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:14.992956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:14.994278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:14.995174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:14.995462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:14.995739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] PATH_INFO: `/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2/users/e5d251fc19f74731aa9ee99af8a7b3ef/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:14.997698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf063a6ffc6e4ab08f3bbe3919becbd2, user_id=e5d251fc19f74731aa9ee99af8a7b3ef, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:14.998418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fddfc-de4f-441f-8a7e-9d379bebe682 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.000491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b158a-d49f-464d-b2df-3befd19e5542 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.004661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.005322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.005603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.005882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.006599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.008532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.011056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 907/1806] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:14 2022] PUT /identity/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2/users/e5d251fc19f74731aa9ee99af8a7b3ef/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:15.022577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.022998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.023160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.029272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 900/1807] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:14 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:15.032990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.042389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:15.044125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a730e-50dc-4762-aa06-1b5918a67b7a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.054549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] MFA Rules not processed for user `e5d251fc19f74731aa9ee99af8a7b3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:15.055756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:15.104094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.104760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.105048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.105338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.106099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.107720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.118598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 901/1808] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:15.122699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.125164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 908/1809] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:15.132402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d64cf-21a6-4067-9a68-92681500a6f8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.137006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.137763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.137872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.138181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.138460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] PATH_INFO: `/v3/projects/afea21afa03744c582dedd2d9ec45754/users/7374a738d65d4da3812fc1c3a7d39bc5/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.155864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] RBAC: Authorizing `identity:create_grant(project_id=afea21afa03744c582dedd2d9ec45754, user_id=7374a738d65d4da3812fc1c3a7d39bc5, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.158664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.173431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 902/1810] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:15 2022] PUT /identity/v3/projects/afea21afa03744c582dedd2d9ec45754/users/7374a738d65d4da3812fc1c3a7d39bc5/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:15.177393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.193835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3ba7a-387e-47f5-b397-5dae41762548 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.244843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003aafc-3842-4668-af6c-862ac444398a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.249829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.250717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.250850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.251058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.251363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.252101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.252101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.252280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.252970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.254866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.272110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 903/1811] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:15.276489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.293803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfb74a-720d-49f0-9bf5-d4c7998546c3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.299993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.301614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.302220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.302650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] PATH_INFO: `/v3/projects/afea21afa03744c582dedd2d9ec45754/users/7374a738d65d4da3812fc1c3a7d39bc5/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.305919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=afea21afa03744c582dedd2d9ec45754, user_id=7374a738d65d4da3812fc1c3a7d39bc5, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.310098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c72f55-bacd-4a0f-a5cb-2969e9ec14bc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.326625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 904/1812] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:15 2022] PUT /identity/v3/projects/afea21afa03744c582dedd2d9ec45754/users/7374a738d65d4da3812fc1c3a7d39bc5/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:15.329994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.331468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.337585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.340199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:15.341224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.341969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.346952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 909/1813] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:15.376790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] MFA Rules not processed for user `7374a738d65d4da3812fc1c3a7d39bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:15.378877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:15.398979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.400467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:15.414823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afe72e-eda7-4a68-9647-c89f1ad294f5 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.444445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:15.484621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.485351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.485705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.486127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.514361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 905/1814] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:15.524554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:15.539647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4928129-0f68-4cf9-a250-c746233eedbc None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:15.544934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:15.546171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:15.546818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:15.547276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:15.571906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.573749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.583197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:15.586429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:15.599055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 906/1815] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:15.611332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 910/1816] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:16.064249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:16.085647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31cdc-81e5-42b5-a779-edde222145ca service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:16.089635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:16.090486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:16.090802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:16.091109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:16.177216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:16.178733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210eb314-6cc4-4b64-958d-f31adb640fd6 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:16.192561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 907/1817] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:16.916745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:16.925460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeba80a-bfc9-4ab1-8011-b70fac89c866 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:16.929737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:16.930510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:16.930803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:16.931082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:16.994966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:16.996438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead568a-cc50-47bd-a331-589822a73616 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:17.008325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 911/1818] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:17.486378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:17.486893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:17.486983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:17.488388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 908/1819] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:52:17.496125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:17.496125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:17.496340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:17.531102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:17.532752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:17.596607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:17.618261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 912/1820] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:52:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:17.630981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:17.632701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:17.651764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c88f66-28e0-4867-9f43-c0d64d90425c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:17.661461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:17.662361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:17.662716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:17.663152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:17.673048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:17.674722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc0c91-2eff-446e-b3f6-620f34481ee2 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:17.685784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 909/1821] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:19.093419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:19.276006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96684e97-34af-4b09-a5bc-7432d122939f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:19.300860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:19.301980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:19.302425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:19.302798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] PATH_INFO: `/v3/users/c1b6a2d2249b4c6489995e5206e9e420` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:19.305103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b6a2d2249b4c6489995e5206e9e420)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:19.315667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:19.455323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b6a2d2249b4c6489995e5206e9e420'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:19.478675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b6a2d2249b4c6489995e5206e9e420'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:19.484193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b6a2d2249b4c6489995e5206e9e420'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:19.484193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b6a2d2249b4c6489995e5206e9e420'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:19.509450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b6a2d2249b4c6489995e5206e9e420'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:19.546214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 913/1822] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:19 2022] DELETE /identity/v3/users/c1b6a2d2249b4c6489995e5206e9e420 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:19.550461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:19.551441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:19.551869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:19.619113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:19.620503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:19.721639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 910/1823] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:19.731896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:19.733484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:19.813439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e1fc6-edc8-4eba-bd6b-1d824c2b6f54 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:19.822480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:19.824653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:19.824771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:19.826396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:19.857000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:19.858525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:19.894561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 914/1824] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:20.678706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:20.680176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:20.697133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d8725-9115-45d1-a14e-67f51eea7c12 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:20.701996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:20.702860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:20.703351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:20.703642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] PATH_INFO: `/v3/projects/4f52bfa5c8304ec3b5249a6dcceac5ac` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:20.704871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f52bfa5c8304ec3b5249a6dcceac5ac)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:20.706861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:20.771431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:20.795585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f52bfa5c8304ec3b5249a6dcceac5ac', 'request_id': 'req-6a2378fe-679f-4a4f-b133-15ea9ad8206e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:20.802684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f52bfa5c8304ec3b5249a6dcceac5ac', 'request_id': 'req-6a2378fe-679f-4a4f-b133-15ea9ad8206e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:20.814882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f52bfa5c8304ec3b5249a6dcceac5ac', 'request_id': 'req-6a2378fe-679f-4a4f-b133-15ea9ad8206e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:20.834335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] The token cache is being invalidate because project 4f52bfa5c8304ec3b5249a6dcceac5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:20.834780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:20.835692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:20.842215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 911/1825] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:20 2022] DELETE /identity/v3/projects/4f52bfa5c8304ec3b5249a6dcceac5ac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:20.842946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:20.857888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff74c5-412e-4ad0-a12a-63025bbe0b72 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:20.863869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:20.958177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:20.959159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:20.959443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:20.959712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] PATH_INFO: `/v3/users/46aa8039b0f74e7a870cc14a13ff4bad` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:20.961444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46aa8039b0f74e7a870cc14a13ff4bad)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:20.963623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:21.028358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46aa8039b0f74e7a870cc14a13ff4bad'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.038355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46aa8039b0f74e7a870cc14a13ff4bad'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.039380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46aa8039b0f74e7a870cc14a13ff4bad'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.041246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46aa8039b0f74e7a870cc14a13ff4bad'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.059261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46aa8039b0f74e7a870cc14a13ff4bad'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.093403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 915/1826] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:20 2022] DELETE /identity/v3/users/46aa8039b0f74e7a870cc14a13ff4bad => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:21.454621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:21.456322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:21.487116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2378fe-679f-4a4f-b133-15ea9ad8206e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:21.581425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:21.582329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:21.582648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:21.582926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] PATH_INFO: `/v3/projects/6aaf99766b494f0e8c3708f5fb73967a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:21.584129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aaf99766b494f0e8c3708f5fb73967a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:21.586179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:21.634525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:21.636344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:21.641081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aaf99766b494f0e8c3708f5fb73967a', 'request_id': 'req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.649881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aaf99766b494f0e8c3708f5fb73967a', 'request_id': 'req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.657006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aaf99766b494f0e8c3708f5fb73967a', 'request_id': 'req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.675422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] The token cache is being invalidate because project 6aaf99766b494f0e8c3708f5fb73967a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:21.675875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.677078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.678928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 912/1827] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:21 2022] DELETE /identity/v3/projects/6aaf99766b494f0e8c3708f5fb73967a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:21.682923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:21.684498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:21.729778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58f77e-4ba8-47f2-8620-fb7c7c552d18 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:21.747861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:21.766179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2b1b-58c4-4b76-85d2-eea097e511f8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:21.772308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:21.773533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:21.773882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:21.774249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] PATH_INFO: `/v3/users/e5d251fc19f74731aa9ee99af8a7b3ef` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:21.775532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d251fc19f74731aa9ee99af8a7b3ef)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:21.777429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:21.777617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:21.792385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:21.793049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:21.793337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:21.793985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:21.795831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:21.829517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d251fc19f74731aa9ee99af8a7b3ef'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.841561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d251fc19f74731aa9ee99af8a7b3ef'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.842275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d251fc19f74731aa9ee99af8a7b3ef'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.842894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d251fc19f74731aa9ee99af8a7b3ef'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.850988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d251fc19f74731aa9ee99af8a7b3ef'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:21.862144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:21.863542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd0cee-0f87-478d-832b-840756b62c3d service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:21.865415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 913/1828] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:21 2022] DELETE /identity/v3/users/e5d251fc19f74731aa9ee99af8a7b3ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:21.873691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 916/1829] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:22.339414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:22.341997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:22.454683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a1e55-edb2-4180-b0f0-af7505265786 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:22.462152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:22.462877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:22.463142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:22.463418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] PATH_INFO: `/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:22.464655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf063a6ffc6e4ab08f3bbe3919becbd2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:22.467776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:22.548402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf063a6ffc6e4ab08f3bbe3919becbd2', 'request_id': 'req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:22.571386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf063a6ffc6e4ab08f3bbe3919becbd2', 'request_id': 'req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:22.577720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf063a6ffc6e4ab08f3bbe3919becbd2', 'request_id': 'req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:22.592348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:22.605635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] The token cache is being invalidate because project bf063a6ffc6e4ab08f3bbe3919becbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:22.605928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:22.606768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-5d2c2eb0-16a9-4e68-9b72-45b515cc79d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:22.609066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 914/1830] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:22 2022] DELETE /identity/v3/projects/bf063a6ffc6e4ab08f3bbe3919becbd2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:22.615139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:22.615608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:22.616041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:22.636242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:22.637659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:22.727993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 917/1831] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:22.737351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de413-84da-4597-a78e-529dda529a5f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:22.738112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de413-84da-4597-a78e-529dda529a5f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:22.738329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85de413-84da-4597-a78e-529dda529a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:22.767107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b85de413-84da-4597-a78e-529dda529a5f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:22.768084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b85de413-84da-4597-a78e-529dda529a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:22.781291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 915/1832] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:22.785623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:22.805498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f472ed5-5ff5-4fff-9c78-117a07a0bc41 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:22.814746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:22.815540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:22.815872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:22.816681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:22.818000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:22.818493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:22.821512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:22.848905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 918/1833] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:22.849242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85de413-84da-4597-a78e-529dda529a5f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:22.863217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85de413-84da-4597-a78e-529dda529a5f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:22.871843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:22.872635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:22.873179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:22.873321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:22.875049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:22.877698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:22.920268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 916/1834] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:22 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:22.924277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:22.942454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648874-2239-4437-ae97-ed042fc1d5ba admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.059382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.060065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.060350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.060628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.061441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.063393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.086622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 919/1835] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:22 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:23.090664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.101235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1673f9-e55a-4793-8551-ad30d1b059df admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.106295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.106962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.107253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.107533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.108264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.109878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.120561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 917/1836] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:23.124570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.136122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008957a-f7b8-48c5-bb2c-a861abbd6330 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.140787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.141442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.141727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.142035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.142762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.144480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf19bc-d68f-405b-984e-0f4bb8742fab admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.157985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 920/1837] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:23.159600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.169603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243726d-7d3b-4d05-b78c-f86f0415fb8e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.177592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.178744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.179028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.179313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] PATH_INFO: `/v3/projects/927a852218194d69aebafc77ec70f770/users/ef1d6a9158cd4ff8984edbdcd6e47456/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.195472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=927a852218194d69aebafc77ec70f770, user_id=ef1d6a9158cd4ff8984edbdcd6e47456, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.198471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.213013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 918/1838] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:23 2022] PUT /identity/v3/projects/927a852218194d69aebafc77ec70f770/users/ef1d6a9158cd4ff8984edbdcd6e47456/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:23.231557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.231928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.232201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.240250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:23.273744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] MFA Rules not processed for user `ef1d6a9158cd4ff8984edbdcd6e47456`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:23.275108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:23.381543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 921/1839] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:23.383056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.465103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ba764-47b7-4452-9877-616e36fbe0db admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.470222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.470903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.471252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.471592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.472334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.474321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.498834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 919/1840] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:23 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:23.502792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.514487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76582-0a67-460f-b473-e911dca9b724 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.569868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.570568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.570857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.571138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.571836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.573757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.593472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 922/1841] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:23 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:23.597473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.607647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55798007-4c37-48e5-8e6c-4ac63a15d303 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.612160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.612808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.613089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.613362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.614631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.616223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.626952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 920/1842] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:23.631555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.641262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d866d33-2870-4235-af6b-b2aa0c613938 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.646222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.647090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.647373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.647648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] PATH_INFO: `/v3/projects/678c93f86ec14a71a432c21a8f2e244e/users/e06e4285f2214724ae14ad3570ce3a17/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.662249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678c93f86ec14a71a432c21a8f2e244e, user_id=e06e4285f2214724ae14ad3570ce3a17, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.665034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.678065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 923/1843] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:23 2022] PUT /identity/v3/projects/678c93f86ec14a71a432c21a8f2e244e/users/e06e4285f2214724ae14ad3570ce3a17/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:23.682015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.692403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299299a-d260-48e4-83ca-549209671a52 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.749155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.749868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.750074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.750355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.751075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.752715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe16dac-94c0-4a34-99f1-78323d165916 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.762993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 921/1844] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:23.767244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.777422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb424de0-67a0-47ff-927e-0479a76dd809 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:23.782018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:23.782893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.783175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.783452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] PATH_INFO: `/v3/projects/678c93f86ec14a71a432c21a8f2e244e/users/e06e4285f2214724ae14ad3570ce3a17/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.785402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678c93f86ec14a71a432c21a8f2e244e, user_id=e06e4285f2214724ae14ad3570ce3a17, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:23.788145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:23.798358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 924/1845] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:23 2022] PUT /identity/v3/projects/678c93f86ec14a71a432c21a8f2e244e/users/e06e4285f2214724ae14ad3570ce3a17/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:23.808922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:23.809220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:23.809505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:23.815343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:23.837750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] MFA Rules not processed for user `e06e4285f2214724ae14ad3570ce3a17`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:23.839103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:23.920211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 922/1846] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:23.939072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:23.941785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:23.997035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:24.012959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08d03e-3207-45de-ad31-fcb9f3108887 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:24.018225 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:24.019000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:24.019579 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:24.020211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:24.039151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:24.040602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:24.054822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 925/1847] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:24.149602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:24.151074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:24.163336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eba65-beb5-489d-b047-3d4973beb743 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:24.167002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:24.167655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:24.167936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:24.168207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:24.225743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:24.227226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd10f7-4c73-4f7b-b533-40203bb3960b service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:24.235965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 923/1848] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:24.510043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:24.571984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed462-e44f-4093-ba5d-e20a5f2acfd3 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:24.576592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:24.577573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:24.577738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:24.578046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] PATH_INFO: `/v3/users/ef1d6a9158cd4ff8984edbdcd6e47456` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:24.579353 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1d6a9158cd4ff8984edbdcd6e47456)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:24.581293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:24.638063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1d6a9158cd4ff8984edbdcd6e47456'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:24.645675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1d6a9158cd4ff8984edbdcd6e47456'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:24.646345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1d6a9158cd4ff8984edbdcd6e47456'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:24.646954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1d6a9158cd4ff8984edbdcd6e47456'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:24.654204 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1d6a9158cd4ff8984edbdcd6e47456'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:24.671822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 926/1849] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:24 2022] DELETE /identity/v3/users/ef1d6a9158cd4ff8984edbdcd6e47456 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:24.676566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:24.676858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:24.677245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:24.694374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:24.695608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:24.753867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 924/1850] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:24.763390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:24.764706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:24.821396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1c4c7-4214-457c-b9d4-a3f91ad6e7b6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:24.825051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:24.825690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:24.826004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:24.826285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:24.835781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:24.837219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:24.846594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 927/1851] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:25.147165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:25.148960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:25.178986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c450e64-bbc3-4f1d-b4e7-9387b71c3937 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:25.184022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:25.185018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:25.185375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:25.185718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] PATH_INFO: `/v3/projects/927a852218194d69aebafc77ec70f770` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:25.187196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927a852218194d69aebafc77ec70f770)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:25.189685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:25.223069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:25.259758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927a852218194d69aebafc77ec70f770', 'request_id': 'req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.266204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927a852218194d69aebafc77ec70f770', 'request_id': 'req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.273314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927a852218194d69aebafc77ec70f770', 'request_id': 'req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.287279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] The token cache is being invalidate because project 927a852218194d69aebafc77ec70f770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:25.287575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.288223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.289901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 925/1852] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:25 2022] DELETE /identity/v3/projects/927a852218194d69aebafc77ec70f770 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:25.293862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:25.304273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21568157-eee5-4fcc-9329-121bdf7ee3d8 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:25.309132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:25.363373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:25.364877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:25.365226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:25.365566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] PATH_INFO: `/v3/users/e06e4285f2214724ae14ad3570ce3a17` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:25.366881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06e4285f2214724ae14ad3570ce3a17)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:25.368866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:25.412723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06e4285f2214724ae14ad3570ce3a17'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.419689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06e4285f2214724ae14ad3570ce3a17'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.420362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06e4285f2214724ae14ad3570ce3a17'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.421003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06e4285f2214724ae14ad3570ce3a17'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.428801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-55e82266-bff7-4959-baac-a5d2db56de93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06e4285f2214724ae14ad3570ce3a17'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.446747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 928/1853] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:25 2022] DELETE /identity/v3/users/e06e4285f2214724ae14ad3570ce3a17 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:25.770106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:25.771720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:25.784902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9642ec1-15cf-4466-ad55-6c45ea3d35b0 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:25.856087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:25.856953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:25.857232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:25.857505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] PATH_INFO: `/v3/projects/678c93f86ec14a71a432c21a8f2e244e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:25.858717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=678c93f86ec14a71a432c21a8f2e244e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:25.860629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:25.912484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678c93f86ec14a71a432c21a8f2e244e', 'request_id': 'req-1b14b64a-9161-4d47-b683-fad03c69bf53'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.918516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678c93f86ec14a71a432c21a8f2e244e', 'request_id': 'req-1b14b64a-9161-4d47-b683-fad03c69bf53'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.924015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678c93f86ec14a71a432c21a8f2e244e', 'request_id': 'req-1b14b64a-9161-4d47-b683-fad03c69bf53'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.937202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] The token cache is being invalidate because project 678c93f86ec14a71a432c21a8f2e244e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:25.937490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.938144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1b14b64a-9161-4d47-b683-fad03c69bf53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:25.939929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 926/1854] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:25 2022] DELETE /identity/v3/projects/678c93f86ec14a71a432c21a8f2e244e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:25.944992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:25.945291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:25.945658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:25.964097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:25.965312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:26.026768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 929/1855] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:26.034491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.034781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.035061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.055942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:26.057202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:26.065742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 927/1856] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:26.070102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.081729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72ac5f-4ba3-47d6-9402-6fe4d7fa3db0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.088194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.088869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.089156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.089437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.090363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:26.090677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.092422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.106797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 930/1857] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:26.110846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.121941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde81418-4ae0-4e82-aecc-d8de6181a6a7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.127118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.127782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.128062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.128333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.129039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.131039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.158183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 928/1858] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:26 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:26.163064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.173139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe827c-2bbe-46b9-a3cd-5f509112633c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.185751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:26.247893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.248480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.248861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.249136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.249855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.252034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.276801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 931/1859] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:26 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:26.281570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.299267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10ea16-e60c-483b-9558-bd727b090655 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.304829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.306922 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.307241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.307701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.308846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.311627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.319471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:26.329238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 929/1860] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:26.332255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.349484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d42138-93e9-48b3-82f5-693bb56ea20f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.354464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.355138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.355427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.355706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.356437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.358088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05fde-ff75-4559-a20b-20808bd9a30c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.369686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 932/1861] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:26.373222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.384909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27150b30-0b2c-40ce-a159-f3ec6f19ee59 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.396754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.397660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.397940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.398238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] PATH_INFO: `/v3/projects/24bd30764d84457ca7f34e9127a89200/users/1923be771a1c4f179d3eba024c6bd787/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.414927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bd30764d84457ca7f34e9127a89200, user_id=1923be771a1c4f179d3eba024c6bd787, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.417909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.432211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 930/1862] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:26 2022] PUT /identity/v3/projects/24bd30764d84457ca7f34e9127a89200/users/1923be771a1c4f179d3eba024c6bd787/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:26.445672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.446189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.446653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.480502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] MFA Rules not processed for user `1923be771a1c4f179d3eba024c6bd787`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:26.481715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:26.561517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 933/1863] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:26.576909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:26.579050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:26.653539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663127-89be-4f34-904e-f901151f44c3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:26.657528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:26.658520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:26.658933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:26.659310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:26.670081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:26.671872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b5eba-cfef-4ed0-88fe-46e2a63c6f68 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:26.696935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 931/1864] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:27.960624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:27.963314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:28.067229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a82d37-fa85-4f69-9211-9473e6027425 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:28.073386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:28.074693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:28.074984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:28.075286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] PATH_INFO: `/v3/users/f0985089eee647acb6b5396d88e12f12` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:28.077064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0985089eee647acb6b5396d88e12f12)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:28.079104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:28.124568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:28.171006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0985089eee647acb6b5396d88e12f12'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:28.183897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0985089eee647acb6b5396d88e12f12'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:28.185483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0985089eee647acb6b5396d88e12f12'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:28.187312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0985089eee647acb6b5396d88e12f12'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:28.202334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0985089eee647acb6b5396d88e12f12'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:28.260987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 934/1865] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:27 2022] DELETE /identity/v3/users/f0985089eee647acb6b5396d88e12f12 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:28.266389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:28.267368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:28.267963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:28.323565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:28.324754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:28.435732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 932/1866] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:28.446804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:28.448419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:28.542795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d65da2-78a2-4368-b084-544f19296bc3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:28.547585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:28.549386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:28.549680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:28.549980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:28.562879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:28.564482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:28.576462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 935/1867] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:28.973470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:28.975954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:29.002268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6deaa10-8d74-4fbb-a361-7a4bd9160f21 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:29.024556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:29.025624 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:29.025917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:29.026227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] PATH_INFO: `/v3/projects/b4e0d7c21dc9483383fae87c5990262f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:29.031482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e0d7c21dc9483383fae87c5990262f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:29.033511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:29.140219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e0d7c21dc9483383fae87c5990262f', 'request_id': 'req-6da70fb0-42fd-4a71-a6fd-c3080baacaca'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.148539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e0d7c21dc9483383fae87c5990262f', 'request_id': 'req-6da70fb0-42fd-4a71-a6fd-c3080baacaca'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.156105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e0d7c21dc9483383fae87c5990262f', 'request_id': 'req-6da70fb0-42fd-4a71-a6fd-c3080baacaca'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.170202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] The token cache is being invalidate because project b4e0d7c21dc9483383fae87c5990262f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:29.170530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.171053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.172839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 933/1868] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:28 2022] DELETE /identity/v3/projects/b4e0d7c21dc9483383fae87c5990262f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:29.177185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:29.188371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bda47a-6a4d-4967-ae51-d492e13d35e3 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:29.258898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:29.259777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:29.260058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:29.260333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] PATH_INFO: `/v3/users/b7eb778a4352490590e4945c8beb92ec` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:29.261524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7eb778a4352490590e4945c8beb92ec)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:29.263530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:29.319050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7eb778a4352490590e4945c8beb92ec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.320356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:29.327821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7eb778a4352490590e4945c8beb92ec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.328652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7eb778a4352490590e4945c8beb92ec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.329468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7eb778a4352490590e4945c8beb92ec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.337729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7eb778a4352490590e4945c8beb92ec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:29.358262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 936/1869] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:29 2022] DELETE /identity/v3/users/b7eb778a4352490590e4945c8beb92ec => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:29.916393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:29.917973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:29.930323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:29.936374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da70fb0-42fd-4a71-a6fd-c3080baacaca admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:29.999129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:30.000149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.000525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.000866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] PATH_INFO: `/v3/projects/6519eb653f68444286b7b3174ea0cea5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.002153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6519eb653f68444286b7b3174ea0cea5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:30.004199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:30.059004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6519eb653f68444286b7b3174ea0cea5', 'request_id': 'req-6091216a-7270-493e-93e3-9d9fda842b35'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.067318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6519eb653f68444286b7b3174ea0cea5', 'request_id': 'req-6091216a-7270-493e-93e3-9d9fda842b35'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.073994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6519eb653f68444286b7b3174ea0cea5', 'request_id': 'req-6091216a-7270-493e-93e3-9d9fda842b35'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.088607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] The token cache is being invalidate because project 6519eb653f68444286b7b3174ea0cea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:30.088901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.089567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.091150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 934/1870] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:29 2022] DELETE /identity/v3/projects/6519eb653f68444286b7b3174ea0cea5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:30.095095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:30.096534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:30.110205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4ff8b-ad7c-4cf6-8c82-c813549131d8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:30.177421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:30.180024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.180491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.180823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] PATH_INFO: `/v3/users/7374a738d65d4da3812fc1c3a7d39bc5` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.182218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7374a738d65d4da3812fc1c3a7d39bc5)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:30.184509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:30.243336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7374a738d65d4da3812fc1c3a7d39bc5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.248417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7374a738d65d4da3812fc1c3a7d39bc5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.249119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7374a738d65d4da3812fc1c3a7d39bc5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.250081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7374a738d65d4da3812fc1c3a7d39bc5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.258070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c43a2fa3-9696-40d3-928c-eff106231335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7374a738d65d4da3812fc1c3a7d39bc5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.276290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 937/1871] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:30 2022] DELETE /identity/v3/users/7374a738d65d4da3812fc1c3a7d39bc5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:30.557518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:30.558986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:30.582965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091216a-7270-493e-93e3-9d9fda842b35 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:30.658345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:30.659219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.659500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.659774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] PATH_INFO: `/v3/projects/afea21afa03744c582dedd2d9ec45754` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.661205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=afea21afa03744c582dedd2d9ec45754)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:30.663165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:30.703373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:30.714149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afea21afa03744c582dedd2d9ec45754', 'request_id': 'req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.720715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afea21afa03744c582dedd2d9ec45754', 'request_id': 'req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.726492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afea21afa03744c582dedd2d9ec45754', 'request_id': 'req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.741296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] The token cache is being invalidate because project afea21afa03744c582dedd2d9ec45754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:30.741592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.742257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f453c8ce-a0b1-467a-8a52-0934bf6ba5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:30.744073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 935/1872] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:30 2022] DELETE /identity/v3/projects/afea21afa03744c582dedd2d9ec45754 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:30.750748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947aca-7132-4572-98b5-853111211d58 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.751158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947aca-7132-4572-98b5-853111211d58 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.751543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c947aca-7132-4572-98b5-853111211d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.770215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7c947aca-7132-4572-98b5-853111211d58 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:30.771525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7c947aca-7132-4572-98b5-853111211d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:30.814207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:30.846458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 938/1873] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:30.846609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.846609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.846609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.865523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:30.866774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:30.874913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 936/1874] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:30.878871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947aca-7132-4572-98b5-853111211d58 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:30.895538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c947aca-7132-4572-98b5-853111211d58 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:30.900208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:30.900879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.901162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.901476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.902233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:30.902543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:30.904149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:30.916456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 939/1875] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:52:30.920878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:30.930958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbaa2b-8c66-46eb-a04e-2dd2bf7bf44b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:30.935740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:30.936396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:30.936711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:30.936995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:30.937728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:30.939766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:30.969757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 937/1876] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:30 2022] POST /identity/v3/projects => generated 395 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:30.975816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:30.989344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593c168-00fd-4be2-8840-55707fa63330 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.060903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.061669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.062067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.062348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.063052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.064991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.091305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 940/1877] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:30 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:31.092736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.104932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bc89-318d-4490-a6c7-bdc1b7cead3d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.110096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.110804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.111097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.111355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.112237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.114703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.134541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 938/1878] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:31.134837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.144485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ecc52-82fa-496f-a420-c5ab2d09bc19 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.149618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.150327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.150609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.150884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.151614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.153226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65179e39-11a8-4f29-9cc2-2d5bf452a1b1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.165132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 941/1879] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:31.169434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.182395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7ca9f-35a9-4e85-a062-c53b4f92d084 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.189330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.190521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.190870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.191312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] PATH_INFO: `/v3/projects/1e9e00c110284d1b80b322ec3d2be04d/users/86803721d87a4d378aaae699f37b5add/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.208912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9e00c110284d1b80b322ec3d2be04d, user_id=86803721d87a4d378aaae699f37b5add, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.211962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.227592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 939/1880] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:31 2022] PUT /identity/v3/projects/1e9e00c110284d1b80b322ec3d2be04d/users/86803721d87a4d378aaae699f37b5add/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:31.245233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.246950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.247317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.295789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] MFA Rules not processed for user `86803721d87a4d378aaae699f37b5add`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:31.297058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:31.419950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 942/1881] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:31.426159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.495896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c89ea7-e24d-4cbe-9663-0ee4d5484c25 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.502613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.503668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.504297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.504726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.505633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.507971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.538246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 940/1882] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:31 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:31.542217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.553533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a8d83-2e1e-4a32-81fa-bc9fe366aef8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.592617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:31.614119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.614740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.615028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.615939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.616653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.618689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.639314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 943/1883] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:31 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:31.643189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.655190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fc2b9-5caf-4467-b9ec-90c2d75760ed admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.659994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.661218 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.661504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.661782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.662536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.664213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.675347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 941/1884] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:31.679489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.689689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b53a7-01b3-4649-91aa-e479acb8b69d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.695489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.696359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.696645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.696923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] PATH_INFO: `/v3/projects/06df4e87d142496bb6c893528a1144a7/users/037fa274c90c42b284cecac64aef3da8/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.712544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=06df4e87d142496bb6c893528a1144a7, user_id=037fa274c90c42b284cecac64aef3da8, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.715333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.729196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 944/1885] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:31 2022] PUT /identity/v3/projects/06df4e87d142496bb6c893528a1144a7/users/037fa274c90c42b284cecac64aef3da8/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:31.733153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.743480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d31a55-be02-4c95-a0e1-f92f9bc95ffe admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.786656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:31.820537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.821473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.821944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.822428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.823312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.825376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3be68d-713b-4a0f-bf0c-88e7cb5c147b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.841637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:31.847586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 942/1886] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:31.853625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb3c864-ea14-4431-a3f3-3c4d9380e0ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:31.859280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:31.863156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.863545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.863976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] PATH_INFO: `/v3/projects/06df4e87d142496bb6c893528a1144a7/users/037fa274c90c42b284cecac64aef3da8/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.867668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06df4e87d142496bb6c893528a1144a7, user_id=037fa274c90c42b284cecac64aef3da8, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:31.870575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:31.892536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 945/1887] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:31 2022] PUT /identity/v3/projects/06df4e87d142496bb6c893528a1144a7/users/037fa274c90c42b284cecac64aef3da8/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:31.895899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:31.896400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:31.896832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:31.928607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] MFA Rules not processed for user `037fa274c90c42b284cecac64aef3da8`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:31.929841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:32.006770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 943/1888] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:32.018442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:32.020778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:32.084078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305cf2d-9721-4928-9a8a-608d303dcbd1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:32.087910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:32.088639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:32.089011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:32.089366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:32.100034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:32.101657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:32.113885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 946/1889] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:32.293468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:32.295059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:32.315552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737843f-07d3-4beb-992a-c51282139eb0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:32.319864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:32.320557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:32.320794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:32.321072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:32.383135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:32.384710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233e8e5-0f5a-4cc8-852e-53be70d762d1 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:32.394791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 944/1890] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:33.179760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:33.183401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:33.324169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650d14-40fc-4419-922e-dd6773705893 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:33.342239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:33.343358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:33.343651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:33.343930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] PATH_INFO: `/v3/users/f60d2e6bddb64ad39ddaf8325ca1cff3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:33.345190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f60d2e6bddb64ad39ddaf8325ca1cff3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:33.347489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:33.417926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:33.441203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f60d2e6bddb64ad39ddaf8325ca1cff3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:33.452646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f60d2e6bddb64ad39ddaf8325ca1cff3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:33.453663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f60d2e6bddb64ad39ddaf8325ca1cff3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:33.454323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f60d2e6bddb64ad39ddaf8325ca1cff3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:33.466050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f60d2e6bddb64ad39ddaf8325ca1cff3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:33.499447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 947/1891] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:33 2022] DELETE /identity/v3/users/f60d2e6bddb64ad39ddaf8325ca1cff3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:33.504519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:33.504968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:33.505141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:33.524480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:33.525911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:33.603752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:33.621140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 945/1892] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:33.631724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:33.633286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:33.696015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4fa1b2-8b9d-49f8-b645-23e5db11c857 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:33.700122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:33.700781 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:33.701068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:33.701371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:33.711004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:33.712477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:33.722243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 948/1893] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:34.273901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:34.275984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:34.293191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c75e7-c913-456f-bdbc-44ecaceae17f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:34.312181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:34.313083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:34.313372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:34.313654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] PATH_INFO: `/v3/projects/67ae1bdc29c44536a14d2918d4e1f61e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:34.314920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ae1bdc29c44536a14d2918d4e1f61e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:34.317632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:34.409170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ae1bdc29c44536a14d2918d4e1f61e', 'request_id': 'req-aff6beb2-3d10-4fb7-998b-12a94eecc36f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.417344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ae1bdc29c44536a14d2918d4e1f61e', 'request_id': 'req-aff6beb2-3d10-4fb7-998b-12a94eecc36f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.424668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ae1bdc29c44536a14d2918d4e1f61e', 'request_id': 'req-aff6beb2-3d10-4fb7-998b-12a94eecc36f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.439526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] The token cache is being invalidate because project 67ae1bdc29c44536a14d2918d4e1f61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:34.439835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.440447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-aff6beb2-3d10-4fb7-998b-12a94eecc36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.446029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 946/1894] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:34 2022] DELETE /identity/v3/projects/67ae1bdc29c44536a14d2918d4e1f61e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:34.446164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:34.455658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d320485-ec60-4054-94b7-376219892e49 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:34.532576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:34.533501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:34.533788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:34.535970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] PATH_INFO: `/v3/users/6ce746b1ec8f46ca8d01f93363646984` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:34.537582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce746b1ec8f46ca8d01f93363646984)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:34.540273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:34.614219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce746b1ec8f46ca8d01f93363646984'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.647523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce746b1ec8f46ca8d01f93363646984'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.648423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce746b1ec8f46ca8d01f93363646984'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.649510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce746b1ec8f46ca8d01f93363646984'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.653363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:34.667176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51c09d93-66b7-4b6c-a3dd-c1a611a96784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce746b1ec8f46ca8d01f93363646984'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:34.713573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 949/1895] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:34 2022] DELETE /identity/v3/users/6ce746b1ec8f46ca8d01f93363646984 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:35.005229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855928f7-6172-49ee-be86-84e41454b009 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.007994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855928f7-6172-49ee-be86-84e41454b009 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.008706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855928f7-6172-49ee-be86-84e41454b009 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.015256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 947/1896] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:52:35 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:52:35.026135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.027766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.028575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.065927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:35.069476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:35.237696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 950/1897] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:35.281592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855928f7-6172-49ee-be86-84e41454b009 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:35.284269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-855928f7-6172-49ee-be86-84e41454b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:35.339335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855928f7-6172-49ee-be86-84e41454b009 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:35.344839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:35.345944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.346517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.346817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.361696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:35.364264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:35.381985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 948/1898] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:35.645454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:35.647459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:35.729202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185bd86-241e-42c5-83a6-ade17ab91ca3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:35.734954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:35.736198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.736198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.736649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] PATH_INFO: `/v3/projects/4911e138ddc94697aaf598d2dd43d853` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.737922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4911e138ddc94697aaf598d2dd43d853)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:35.740552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:35.802869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4911e138ddc94697aaf598d2dd43d853', 'request_id': 'req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:35.822479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4911e138ddc94697aaf598d2dd43d853', 'request_id': 'req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:35.830111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:35.831631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:35.837524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4911e138ddc94697aaf598d2dd43d853', 'request_id': 'req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:35.851893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] The token cache is being invalidate because project 4911e138ddc94697aaf598d2dd43d853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:35.852207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:35.852829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5a46196e-b00b-4cf3-a1a2-10ea2039d03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:35.854828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 951/1899] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:35 2022] DELETE /identity/v3/projects/4911e138ddc94697aaf598d2dd43d853 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:35.859280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd5f64-4e07-4c8d-9ba5-37cd5530b156 service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:35.860750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989d26-00ae-4c93-b488-8cf9bb10187f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.861298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989d26-00ae-4c93-b488-8cf9bb10187f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.861837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989d26-00ae-4c93-b488-8cf9bb10187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.870078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:35.889405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ef989d26-00ae-4c93-b488-8cf9bb10187f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:35.890729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ef989d26-00ae-4c93-b488-8cf9bb10187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:35.944202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:35.945394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.945690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.945991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] PATH_INFO: `/v3/users/1923be771a1c4f179d3eba024c6bd787` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.950097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1923be771a1c4f179d3eba024c6bd787)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:35.952377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:35.965635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 952/1900] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:35.973586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a3b3-0800-4f24-acb7-068072cac3d8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:35.973940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a3b3-0800-4f24-acb7-068072cac3d8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:35.973940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1a3b3-0800-4f24-acb7-068072cac3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:35.995862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fca1a3b3-0800-4f24-acb7-068072cac3d8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:35.997358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fca1a3b3-0800-4f24-acb7-068072cac3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.002764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1923be771a1c4f179d3eba024c6bd787'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.007619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 953/1901] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:36.009068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08734c-2075-4422-924b-597028bd7a54 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.010174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08734c-2075-4422-924b-597028bd7a54 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.010581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08734c-2075-4422-924b-597028bd7a54 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.011878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 954/1902] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:52:36.012601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08734c-2075-4422-924b-597028bd7a54 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.019068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1923be771a1c4f179d3eba024c6bd787'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.020839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1923be771a1c4f179d3eba024c6bd787'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.021482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1923be771a1c4f179d3eba024c6bd787'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.023923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:36.030746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08734c-2075-4422-924b-597028bd7a54 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.031752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1923be771a1c4f179d3eba024c6bd787'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.036334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.037001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.037291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.037572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.038337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:36.038643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.040251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e2f11-e930-42c8-96f5-409bbed554d2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.054248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 955/1903] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:36.055391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.055685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.055964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.061190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 949/1904] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:35 2022] DELETE /identity/v3/users/1923be771a1c4f179d3eba024c6bd787 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:36.061190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.062410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.075865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] MFA Rules not processed for user `c27854a0a16544cc9eab7c048fe03833`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:36.077164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.127053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450cbd-b21c-4e0e-9845-5bf3f44b590c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.132080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.132940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.133221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.133495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] PATH_INFO: `/v3/users/ca8e7d3ea52f4a3d942b687633d69ddf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.134710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8e7d3ea52f4a3d942b687633d69ddf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.136663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.140499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 956/1905] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:36.141335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.142671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.157117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e280b-ee9c-457d-b9c4-fa311d61858d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.162422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.163090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.163394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.163659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.164342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.166280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c8490-6569-41b9-b8b0-750cb5828233 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.185347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8e7d3ea52f4a3d942b687633d69ddf'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.192306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 957/1906] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:36.193581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.193868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.194171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.204477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8e7d3ea52f4a3d942b687633d69ddf'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.205286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8e7d3ea52f4a3d942b687633d69ddf'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.205941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8e7d3ea52f4a3d942b687633d69ddf'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.214237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:36.214387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8e7d3ea52f4a3d942b687633d69ddf'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.215719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.226925 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 950/1907] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:36 2022] DELETE /identity/v3/users/ca8e7d3ea52f4a3d942b687633d69ddf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:36.227725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.229007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.294276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:36.310526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 958/1908] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:36.310526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.310526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.318058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:36.323424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f53a3-eeb2-4f65-85ca-52adfe13a589 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.325034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228953-0c11-4264-9e26-2a3473c45ca4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.335696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.337051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.337635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.338124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.340330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.355118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.357081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.367480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 951/1909] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:36.368390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.369782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.390643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.391746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.392159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.392499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.393392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.396020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.423917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 959/1910] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:36.427639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.443421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f074a2-47e0-487f-8bba-3d7e35d16087 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.443970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd670aa-ec4e-4c56-a2ed-ce64565c56f1 service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.449749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.450546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.450898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.451242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.452042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.453220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.453850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.453979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.454185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.454463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.464917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.466433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.468409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 960/1911] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:36.469255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.471083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.477399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 952/1912] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:36.478231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.485178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe44c-af06-423c-a1a3-e465b126d6ac admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.487472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd8735-f1f2-4ee9-b1f2-cb13c1709e58 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.492928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.493558 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.493845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.494194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.494999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.496929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.502792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.503672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.504500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.504782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] PATH_INFO: `/v3/projects/3b495d67b32a4f6da27fe0be6650d2fd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.506169 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b495d67b32a4f6da27fe0be6650d2fd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.509074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.512356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 953/1913] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:36.513579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.526900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ab8f-aa2d-4793-924a-3c8bdd7fb661 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.533375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.534305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.534588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.534867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] PATH_INFO: `/v3/projects/08724cb09c75439a9cda1c54a70d663f/users/72e4059c850a48c4bfdad31c08e5dfa5/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.554344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08724cb09c75439a9cda1c54a70d663f, user_id=72e4059c850a48c4bfdad31c08e5dfa5, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.557090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f3cde-6e5c-4f68-b481-89e2b9063054 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.584123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b495d67b32a4f6da27fe0be6650d2fd', 'request_id': 'req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.589123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 954/1914] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:36 2022] PUT /identity/v3/projects/08724cb09c75439a9cda1c54a70d663f/users/72e4059c850a48c4bfdad31c08e5dfa5/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:36.599846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.600250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.600630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.602521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b495d67b32a4f6da27fe0be6650d2fd', 'request_id': 'req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.611504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b495d67b32a4f6da27fe0be6650d2fd', 'request_id': 'req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.631178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] The token cache is being invalidate because project 3b495d67b32a4f6da27fe0be6650d2fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:36.631530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.632311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.634589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 961/1915] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:36 2022] DELETE /identity/v3/projects/3b495d67b32a4f6da27fe0be6650d2fd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:36.639115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.641877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] MFA Rules not processed for user `72e4059c850a48c4bfdad31c08e5dfa5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:36.644099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.650830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.747840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 955/1916] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:36.753063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.755080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.793290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e924b9-6611-4ac2-bca9-bb2dcf47b7df admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.799593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:36.831318 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:36.852211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f225ddb-a2f3-49a2-b127-a7f6d208e757 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.857275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.858181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.858472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.858751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.859921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.861863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.876341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.877249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.877532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.877808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] PATH_INFO: `/v3/users/ffb172ce90f946b89eea0fac6e7c16fe` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.879719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb172ce90f946b89eea0fac6e7c16fe)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.881875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:36.889887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 956/1917] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:36 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:36.890630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.892575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.938082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb172ce90f946b89eea0fac6e7c16fe'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.945041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb172ce90f946b89eea0fac6e7c16fe'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.945730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb172ce90f946b89eea0fac6e7c16fe'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.946375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb172ce90f946b89eea0fac6e7c16fe'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.953705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb172ce90f946b89eea0fac6e7c16fe'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:36.961995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47b5de-aa8c-4ea7-b4dd-f2c2a263fae9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:36.971152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 962/1918] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:36 2022] DELETE /identity/v3/users/ffb172ce90f946b89eea0fac6e7c16fe => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:36.972085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:36.973546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:36.974093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:36.974553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:36.974847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:36.975130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] PATH_INFO: `/v3/users/86803721d87a4d378aaae699f37b5add` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:36.976472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86803721d87a4d378aaae699f37b5add)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:36.978457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.032048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86803721d87a4d378aaae699f37b5add'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.047108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86803721d87a4d378aaae699f37b5add'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.048035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86803721d87a4d378aaae699f37b5add'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.048875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86803721d87a4d378aaae699f37b5add'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.057324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f5988-4805-4299-b03f-58211f3bc646 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.059802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86803721d87a4d378aaae699f37b5add'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.062605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.063386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.063710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.063941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] PATH_INFO: `/v3/projects/24bd30764d84457ca7f34e9127a89200` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.065157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bd30764d84457ca7f34e9127a89200)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.067673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.080695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 957/1919] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:36 2022] DELETE /identity/v3/users/86803721d87a4d378aaae699f37b5add => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:37.080695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.080695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.105632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.128702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bd30764d84457ca7f34e9127a89200', 'request_id': 'req-b7c6e84e-6d4a-4897-a06c-92b9d349114c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.137270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.143657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bd30764d84457ca7f34e9127a89200', 'request_id': 'req-b7c6e84e-6d4a-4897-a06c-92b9d349114c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.152381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd3cf6-3a52-4c46-8b6f-168818c4c7b6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.152803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bd30764d84457ca7f34e9127a89200', 'request_id': 'req-b7c6e84e-6d4a-4897-a06c-92b9d349114c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.169220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] The token cache is being invalidate because project 24bd30764d84457ca7f34e9127a89200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:37.169757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.170818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b7c6e84e-6d4a-4897-a06c-92b9d349114c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.172911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 963/1920] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:36 2022] DELETE /identity/v3/projects/24bd30764d84457ca7f34e9127a89200 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:37.175566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7c08-58a6-46be-a373-3414de5d1e0d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.176735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7c08-58a6-46be-a373-3414de5d1e0d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.177177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad7c08-58a6-46be-a373-3414de5d1e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.197186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b9ad7c08-58a6-46be-a373-3414de5d1e0d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:37.199035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b9ad7c08-58a6-46be-a373-3414de5d1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.226844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.227592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.227884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.228196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.228923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.231081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.272752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 964/1921] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:37.274129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.274423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.274702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.286125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 958/1922] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:37.287588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.290400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.294279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:37.295481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.303553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21695c-5731-49c8-9668-979fe5d04aaa admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.308328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.309161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.309444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.309718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] PATH_INFO: `/v3/projects/45fad236c8b04df6889bb6321643a848` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.310952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fad236c8b04df6889bb6321643a848)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.312917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 965/1923] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:37.313047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.313826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.316357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.364787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fad236c8b04df6889bb6321643a848', 'request_id': 'req-14beb870-3e75-4664-8860-8afbf01c7299'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.371370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fad236c8b04df6889bb6321643a848', 'request_id': 'req-14beb870-3e75-4664-8860-8afbf01c7299'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.376688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f2a94-dc27-49fe-99de-4c11bb6bde2c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.377535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fad236c8b04df6889bb6321643a848', 'request_id': 'req-14beb870-3e75-4664-8860-8afbf01c7299'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.393002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] The token cache is being invalidate because project 45fad236c8b04df6889bb6321643a848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:37.393384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.394137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:37.396860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 959/1924] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:37 2022] DELETE /identity/v3/projects/45fad236c8b04df6889bb6321643a848 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:37.397504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.399022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.434102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.434817 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.435128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.435407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.437561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.459457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.475038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beb870-3e75-4664-8860-8afbf01c7299 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.480155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.481008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.481451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.481730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.482492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.484118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f929ac-6f06-434f-9663-0ea464a9e7d0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.495492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 960/1925] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:37.497048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.497505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.497942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.515264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.516718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6aefa-b007-44e3-8716-b47af2205845 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.517817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:37.519091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.527866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 966/1926] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:37.528142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 961/1927] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:37.528882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.529163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.529557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.529838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.533818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.541395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d42d-1e7f-4fb3-b986-bc14635a9dc0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.548873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.550173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.550801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.551435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] PATH_INFO: `/v3/projects/499af771924b4911ac53a06bfd64378c/users/ca13c0daeaf84d25a68582aad2c90cb6/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.554075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:37.555400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.564541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 967/1928] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:37.565308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.575168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3f37-9d5b-40a8-91c9-5d716260eadd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.579588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=499af771924b4911ac53a06bfd64378c, user_id=ca13c0daeaf84d25a68582aad2c90cb6, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.580127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.580892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.581585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.581585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.582876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.583098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:37.583506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.585256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ed11a-3c6f-4dd6-aa0b-35c3ced4c558 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.600779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 968/1929] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:37.602509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.602897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.603402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.608564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 962/1930] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:37 2022] PUT /identity/v3/projects/499af771924b4911ac53a06bfd64378c/users/ca13c0daeaf84d25a68582aad2c90cb6/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:37.609762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.624651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:37.626084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:37.688711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0630f-f17a-4f1d-b99f-8f21a1850ccd admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.696337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.697081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.697373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.697910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.698647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.704397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.704564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 969/1931] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:37.704564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.714659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f19d-e73a-476b-9d70-13c37cdef55b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.720436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.720892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.721384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.721653 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.722568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.724418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.736605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 970/1932] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:37.737590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.738294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 963/1933] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:37.740967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.750079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bfa34-415a-4e41-9136-76dc239971ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.823161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.824249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.824537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.824897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.825814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:37.826604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.827765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.829066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e977c4-d463-41ac-a7d4-cb725169c32a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.834427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.835320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.835600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.835881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] PATH_INFO: `/v3/projects/499af771924b4911ac53a06bfd64378c/users/ca13c0daeaf84d25a68582aad2c90cb6/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.838231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499af771924b4911ac53a06bfd64378c, user_id=ca13c0daeaf84d25a68582aad2c90cb6, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.841375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.844378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 971/1934] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:37.846326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.857513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 964/1935] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:37 2022] PUT /identity/v3/projects/499af771924b4911ac53a06bfd64378c/users/ca13c0daeaf84d25a68582aad2c90cb6/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:37.858609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:37.862146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773c93d-59b2-48e6-a5e3-f8a97ebce168 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.889521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.963594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.964205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.964498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.964779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.965484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.966593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:37.967477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c1ec4-22c1-412c-9a5b-19877373953b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:37.973097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e92e26-b8d8-41df-88a8-9f875acea117 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:37.980147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:37.980911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:37.981393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:37.981762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:37.982641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:37.984984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.008607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 972/1936] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:38.008607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.008607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.009379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.016817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 965/1937] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:37 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:38.017578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.049082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] MFA Rules not processed for user `ca13c0daeaf84d25a68582aad2c90cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:38.051874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.103863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f95b6-6e15-4da7-ab12-06a805d650d6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.110108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298df090-3678-4017-8077-43cad487f266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.110987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298df090-3678-4017-8077-43cad487f266 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.111366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298df090-3678-4017-8077-43cad487f266 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.111658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298df090-3678-4017-8077-43cad487f266 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.112749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298df090-3678-4017-8077-43cad487f266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.115014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298df090-3678-4017-8077-43cad487f266 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.125388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 966/1938] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:38.126219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298df090-3678-4017-8077-43cad487f266 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.136836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298df090-3678-4017-8077-43cad487f266 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.141634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.142351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.142632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.142911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.143611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.146355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.147195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 973/1939] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:38.148134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.149633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.164504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304bc00-aac6-417f-b228-afbc6734de98 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.170191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.171057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.171345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.171743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] PATH_INFO: `/v3/projects/1e9e00c110284d1b80b322ec3d2be04d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.171854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 967/1940] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:38.172576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.172944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9e00c110284d1b80b322ec3d2be04d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.174931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.183053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc489-6a75-4885-8ca9-0dbda9895219 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.188190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.188844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.189128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.189405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.190164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.192450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.205496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 968/1941] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:38.206615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.209169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.226492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9e00c110284d1b80b322ec3d2be04d', 'request_id': 'req-9c0dfd12-ae76-4987-8bfb-b275d774872f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.237135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9e00c110284d1b80b322ec3d2be04d', 'request_id': 'req-9c0dfd12-ae76-4987-8bfb-b275d774872f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.246922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9e00c110284d1b80b322ec3d2be04d', 'request_id': 'req-9c0dfd12-ae76-4987-8bfb-b275d774872f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.260482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] The token cache is being invalidate because project 1e9e00c110284d1b80b322ec3d2be04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:38.260778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.261452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.263051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 974/1942] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:38 2022] DELETE /identity/v3/projects/1e9e00c110284d1b80b322ec3d2be04d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:38.263826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.265141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.275590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b06509-9d72-4dd7-a036-e6517ff18c60 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.285563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:38.327649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.328297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.328579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.328851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.330753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.336565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dfd12-ae76-4987-8bfb-b275d774872f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.341308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.342003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.342294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.342575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.343302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.344914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.350450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:38.356824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 975/1943] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:38.357719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.359464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.374964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331368d-be16-4fcc-a9af-7b17e1a28698 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.379916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.380800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.381086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.381365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] PATH_INFO: `/v3/projects/ae263982ccbe4cecb046ae63889618e3/users/7e974e9a7cbe420e960f290086cc75c7/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.397616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae263982ccbe4cecb046ae63889618e3, user_id=7e974e9a7cbe420e960f290086cc75c7, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.397794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.399329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.400352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.410863 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 969/1944] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:38.411549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.412883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.418422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 976/1945] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:38 2022] PUT /identity/v3/projects/ae263982ccbe4cecb046ae63889618e3/users/7e974e9a7cbe420e960f290086cc75c7/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:38.419158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.428798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a6d7d-33a8-4b11-af26-53df298b580b service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.497576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233374a-ef70-4a1c-a1a7-2f9542d779cd admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.500361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.501317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.501666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.502039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] PATH_INFO: `/v3/users/037fa274c90c42b284cecac64aef3da8` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.503352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] RBAC: Authorizing `identity:delete_user(user_id=037fa274c90c42b284cecac64aef3da8)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.503482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.504701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.504990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.505271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.506021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.506334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.507685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1bf18-e65e-492e-ac29-7c2b30552a3c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.518738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 977/1946] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:38.520920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.521212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.521492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.557932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037fa274c90c42b284cecac64aef3da8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.560656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] MFA Rules not processed for user `7e974e9a7cbe420e960f290086cc75c7`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:38.561869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.572664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037fa274c90c42b284cecac64aef3da8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.573436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037fa274c90c42b284cecac64aef3da8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.574238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037fa274c90c42b284cecac64aef3da8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.583501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037fa274c90c42b284cecac64aef3da8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:38.599859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 970/1947] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:38 2022] DELETE /identity/v3/users/037fa274c90c42b284cecac64aef3da8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:38.599859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.599859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.651023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:38.678793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 978/1948] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:38.680368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.682870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.705149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35645669-438c-4a6d-b972-16f13be223e5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.718198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0c941-b41c-4f67-8e19-fc74a672168d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.720284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.721212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.721562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.721941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.722881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.725238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.727418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.728542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.728845 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.729203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] PATH_INFO: `/v3/projects/5c62a63b15b44b05838fb40b8c346953/users/1b65eb5018ba4bf6b04c40627f6dd25f/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.750478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:38.752038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c62a63b15b44b05838fb40b8c346953, user_id=1b65eb5018ba4bf6b04c40627f6dd25f, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.757026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d1f8-cd0a-4781-8bc8-74c3d86de9f0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.772482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 979/1949] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:38.773372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 971/1950] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:38 2022] PUT /identity/v3/projects/5c62a63b15b44b05838fb40b8c346953/users/1b65eb5018ba4bf6b04c40627f6dd25f/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:38.774400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.793399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.793684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.793994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.860587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] MFA Rules not processed for user `1b65eb5018ba4bf6b04c40627f6dd25f`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:38.862644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.887445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ecfc0-cc97-4126-9b22-56480c986535 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.893480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.894196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.894516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.894796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.895774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.898018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.922034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 980/1951] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:38.924450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.938609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6dd68-f24e-44eb-af0e-8270daf3067a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.949328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.950208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.950572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.951500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.951729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:38.953406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:38.966007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 981/1952] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:38.966007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.967831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:38.975489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 972/1953] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:38.976339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:38.990153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc19d-e31e-47f6-961d-d81cd7ebf330 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.993327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1169c-b1b5-44f9-a367-25555e851605 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:38.995633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:38.996517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:38.996798 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:38.997072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] PATH_INFO: `/v3/users/72e4059c850a48c4bfdad31c08e5dfa5` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:38.998745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e4059c850a48c4bfdad31c08e5dfa5)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.000687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.000998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.001739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.002116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.002623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.004388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.006657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.021796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 973/1954] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:39.021796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.021796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.060037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e4059c850a48c4bfdad31c08e5dfa5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.067399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e4059c850a48c4bfdad31c08e5dfa5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.068087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e4059c850a48c4bfdad31c08e5dfa5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.068708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e4059c850a48c4bfdad31c08e5dfa5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.076479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e4059c850a48c4bfdad31c08e5dfa5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.084211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc7826-e2f1-4b98-8873-aaa50c28670b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.088261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.088915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.089199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.089474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.091357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.098199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 982/1955] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:38 2022] DELETE /identity/v3/users/72e4059c850a48c4bfdad31c08e5dfa5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:39.098898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.100280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.105681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.107099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.122801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 974/1956] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:39.123658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.125181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.139676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63058-b39c-43cf-b219-8c79787e6271 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.154002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:39.174546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e86f82-16c3-41ed-9522-861701c04310 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.179888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.180779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.181057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.181332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] PATH_INFO: `/v3/projects/06df4e87d142496bb6c893528a1144a7` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.182718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06df4e87d142496bb6c893528a1144a7)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.183540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:39.185404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.203781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.204704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.204987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.205294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] PATH_INFO: `/v3/projects/3bdacb98af9e40f9a29532b4562a23ee/users/457626fe83264f0a99d242c739879182/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.221765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bdacb98af9e40f9a29532b4562a23ee, user_id=457626fe83264f0a99d242c739879182, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.225130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baef086-65f2-4c07-9445-7afa041786fb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.233933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06df4e87d142496bb6c893528a1144a7', 'request_id': 'req-1eadeecd-c2f9-4aee-b10c-140f32bd4697'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.246242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06df4e87d142496bb6c893528a1144a7', 'request_id': 'req-1eadeecd-c2f9-4aee-b10c-140f32bd4697'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.257790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 975/1957] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:39 2022] PUT /identity/v3/projects/3bdacb98af9e40f9a29532b4562a23ee/users/457626fe83264f0a99d242c739879182/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:39.257790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd237-d48a-4688-b274-3ca6f9115452 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.257790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd237-d48a-4688-b274-3ca6f9115452 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.257790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dd237-d48a-4688-b274-3ca6f9115452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.258237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06df4e87d142496bb6c893528a1144a7', 'request_id': 'req-1eadeecd-c2f9-4aee-b10c-140f32bd4697'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.264206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] The token cache is being invalidate because project 06df4e87d142496bb6c893528a1144a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:39.264489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.265146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.266792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 983/1958] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:39 2022] DELETE /identity/v3/projects/06df4e87d142496bb6c893528a1144a7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:39.267544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.268837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.271635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e42dd237-d48a-4688-b274-3ca6f9115452 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:39.272985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e42dd237-d48a-4688-b274-3ca6f9115452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.346188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 976/1959] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:39.346442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadeecd-c2f9-4aee-b10c-140f32bd4697 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.348968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.349308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.349673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.354355 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.355341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.355641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.356276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] PATH_INFO: `/v3/users/1b65eb5018ba4bf6b04c40627f6dd25f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.357147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b65eb5018ba4bf6b04c40627f6dd25f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.359304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.381156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] MFA Rules not processed for user `457626fe83264f0a99d242c739879182`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:39.382418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.409283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b65eb5018ba4bf6b04c40627f6dd25f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.416587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b65eb5018ba4bf6b04c40627f6dd25f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.417263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b65eb5018ba4bf6b04c40627f6dd25f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.417906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b65eb5018ba4bf6b04c40627f6dd25f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.425475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-101ea2cd-4e63-4a7f-ac7a-187325e597f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b65eb5018ba4bf6b04c40627f6dd25f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:39.436636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:39.445931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 984/1960] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:39 2022] DELETE /identity/v3/users/1b65eb5018ba4bf6b04c40627f6dd25f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:39.447021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 977/1961] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:39.447918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.449779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.451730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40429-4a74-4094-885e-4a0fa20526dc None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.452025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40429-4a74-4094-885e-4a0fa20526dc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.452309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40429-4a74-4094-885e-4a0fa20526dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.469249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-edb40429-4a74-4094-885e-4a0fa20526dc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:39.470495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-edb40429-4a74-4094-885e-4a0fa20526dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.509580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3df8b-278c-45aa-a56b-7a93fe76edf3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.515270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:39.532693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 985/1962] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:39.534017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.534311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.534589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.552821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:39.554048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.564905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 986/1963] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:39.564905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.564905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.567714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.568366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.568651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.568926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.570987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.575797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e728d-de23-407b-94e0-3e9a98550ee0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.580976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.581622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.582478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.582759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.583476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.584338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.585373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.585744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1fdd-117d-4990-a204-3f5dbb92d61f service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.595631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 978/1964] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:39.597126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.597536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.598311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.614686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 987/1965] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:39.615524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.620455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:39.622417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:39.682232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f470fe2-4f3e-45d5-9a4b-84933cca116d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.686660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.687429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.687719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.687990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.696549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 979/1966] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:39.697512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.708441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1de14-d541-4860-8caa-89d384df2aa3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.713900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.714674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.714956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.717067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.717067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.718704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.747550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 980/1967] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/users => generated 299 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:39.748748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.772424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161bb29-3166-4c19-9221-a293bacd0375 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.780279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.781067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.781358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.782179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.782915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:39.783711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.784180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.785969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.786162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.796897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 988/1968] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:39.797809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.808212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7700579-9d76-4908-bd75-6957ffcafbb5 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.812509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 981/1969] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:39.814152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.818971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.819898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.820484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.820918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.821925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.823877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.826658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7ffb-993f-447d-ada1-ca52807b4a96 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.834610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.835699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.836146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.836670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.837861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.840553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 989/1970] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:39.841198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.841570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.849639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:39.854657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71baf6f-db71-476e-9909-750e75f4769f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:39.862871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:39.864082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:39.864373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:39.864726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] PATH_INFO: `/v3/projects/d7470c61398142bfad03a4302d950ec6/users/43f776434ae540b190e5ef531abbc289/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:39.884945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7470c61398142bfad03a4302d950ec6, user_id=43f776434ae540b190e5ef531abbc289, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:39.888258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:39.888463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 982/1971] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/projects => generated 409 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:39.892972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:39.922878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 990/1972] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:39 2022] PUT /identity/v3/projects/d7470c61398142bfad03a4302d950ec6/users/43f776434ae540b190e5ef531abbc289/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:39.927305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.027359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2b9a8-be43-4361-b535-97ad605a6f80 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.032506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.033243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.033620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.033801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820dcbe-6c9d-43ff-a51d-629d9ed9051e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.034160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.035016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.036756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.039524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.040269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.040619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.040986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.042281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.044743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.048165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 991/1973] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:40.048908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.050879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.057140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.065180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6be8e-aa9a-46ca-94b2-a5928632a0e3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.071284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.072158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.072440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.072718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] PATH_INFO: `/v3/projects/08724cb09c75439a9cda1c54a70d663f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.073919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=08724cb09c75439a9cda1c54a70d663f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.075885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.081308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 983/1974] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:39 2022] POST /identity/v3/users => generated 305 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:40.083120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.098511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1131-2202-479f-92fe-a37c1cac2628 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.104820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.106233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.106805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.106805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] PATH_INFO: `/v3/projects/d7470c61398142bfad03a4302d950ec6/users/43f776434ae540b190e5ef531abbc289/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.109381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7470c61398142bfad03a4302d950ec6, user_id=43f776434ae540b190e5ef531abbc289, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.112823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.132400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 984/1975] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:40 2022] PUT /identity/v3/projects/d7470c61398142bfad03a4302d950ec6/users/43f776434ae540b190e5ef531abbc289/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:40.133269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.151332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08724cb09c75439a9cda1c54a70d663f', 'request_id': 'req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.162546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08724cb09c75439a9cda1c54a70d663f', 'request_id': 'req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.170911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08724cb09c75439a9cda1c54a70d663f', 'request_id': 'req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.182261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] The token cache is being invalidate because project 08724cb09c75439a9cda1c54a70d663f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:40.182882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.183558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.185155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 992/1976] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:40 2022] DELETE /identity/v3/projects/08724cb09c75439a9cda1c54a70d663f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:40.188039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.188039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.208538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223a40-1532-450d-b2ba-912af004dfe3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.260658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6e34-4bd3-4e6c-b3f0-b2e46ee6b9cd admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.265780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.266810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.267033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.267165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.267395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] PATH_INFO: `/v3/projects/5c62a63b15b44b05838fb40b8c346953` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.267699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.267984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.268259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.268590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c62a63b15b44b05838fb40b8c346953)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.268977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.271360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.271479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea71954-0a4d-46c5-85de-36d336b9b16a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.272841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.281427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 985/1977] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:40.282645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.282942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.283246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.310874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] MFA Rules not processed for user `43f776434ae540b190e5ef531abbc289`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:40.312089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.326213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c62a63b15b44b05838fb40b8c346953', 'request_id': 'req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.333678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c62a63b15b44b05838fb40b8c346953', 'request_id': 'req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.340324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c62a63b15b44b05838fb40b8c346953', 'request_id': 'req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.350896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] The token cache is being invalidate because project 5c62a63b15b44b05838fb40b8c346953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:40.351185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.351831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.353390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 993/1978] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:40 2022] DELETE /identity/v3/projects/5c62a63b15b44b05838fb40b8c346953 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:40.354248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.355554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.381824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 986/1979] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:40.382701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.383954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.390602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.425482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f2dac-8f2c-4baa-aa55-1ce0e0d306ea admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.430722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.431597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.431882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.432155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] PATH_INFO: `/v3/users/ca13c0daeaf84d25a68582aad2c90cb6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.433351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca13c0daeaf84d25a68582aad2c90cb6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.435299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.451912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76593-3204-4a63-80b5-b73b39ed5688 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.456835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.457663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.458075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.458439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.459356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.461337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1b4ec-7586-45b9-ba32-6f7a02aaaa75 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.473832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 987/1980] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:40.475111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.475406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.475688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.482938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca13c0daeaf84d25a68582aad2c90cb6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.490650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca13c0daeaf84d25a68582aad2c90cb6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.491314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca13c0daeaf84d25a68582aad2c90cb6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.492052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca13c0daeaf84d25a68582aad2c90cb6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.495508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:40.496999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.500377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca13c0daeaf84d25a68582aad2c90cb6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:40.509037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 988/1981] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:40.517451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.518840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.525678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 994/1982] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:40 2022] DELETE /identity/v3/users/ca13c0daeaf84d25a68582aad2c90cb6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:40.526567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.527875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.538372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.587629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2fd14-29bc-482a-84c4-f4cf8521cab9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.605377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceee7a8-9a58-4e60-8075-94399968bfca admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.611476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.612568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.612853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.613894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] PATH_INFO: `/v3/projects/e70612188b15468ca8171f28028e79a0/users/993f5e0b1a824b2fb43d161c4d3b5465/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.632858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70612188b15468ca8171f28028e79a0, user_id=993f5e0b1a824b2fb43d161c4d3b5465, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.635716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb6b00-21a4-468d-8e2d-b9a49bfb6482 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.650520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.651200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.651487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.651748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.653638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 995/1983] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:40 2022] PUT /identity/v3/projects/e70612188b15468ca8171f28028e79a0/users/993f5e0b1a824b2fb43d161c4d3b5465/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:40.655414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.656092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.656092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.663419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.679399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:40.680713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.730829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.732998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e283856-ff1c-4e09-8668-39f43d7ef0a7 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.744093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 989/1984] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:40.753981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 996/1985] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:40.755274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.755568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.760825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.767931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.781220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae597bfd-4678-4b64-bc4a-d18f0f23b31f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.783514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.788162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.789532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.790065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.790488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.791469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:40.792053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.793816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.813144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] MFA Rules not processed for user `993f5e0b1a824b2fb43d161c4d3b5465`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:40.813780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 997/1986] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:40.814441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.815149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.826906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee1a4-42e2-4012-a454-0195bc51f4cc admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.834272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:40.834938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:40.835221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:40.835500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:40.836199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:40.838985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:40.891395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 998/1987] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/projects => generated 359 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:40.896762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.906970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 990/1988] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:40.908239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:40.911068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:40.939896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3ac51-c8c6-4bf9-86ac-581313d82fe8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:40.985675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:40.992153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c7b-ea61-430b-8c14-bf0a00a50467 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.004935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.005527 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.006490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.006769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.007619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.010633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.017265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.018206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.018452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.018815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] PATH_INFO: `/v3/projects/499af771924b4911ac53a06bfd64378c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.020331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499af771924b4911ac53a06bfd64378c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.022316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.046901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 999/1989] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:40 2022] POST /identity/v3/users => generated 280 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:41.047756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.061383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f689738-62dc-43d1-8d67-ca1f478019da admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.066487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.067420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.067420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.067700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.068715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.070751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.101675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1000/1990] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:41.102964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.112818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499af771924b4911ac53a06bfd64378c', 'request_id': 'req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.125807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499af771924b4911ac53a06bfd64378c', 'request_id': 'req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.133700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499af771924b4911ac53a06bfd64378c', 'request_id': 'req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.156247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] The token cache is being invalidate because project 499af771924b4911ac53a06bfd64378c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:41.157164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.158030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.162590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 991/1991] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:40 2022] DELETE /identity/v3/projects/499af771924b4911ac53a06bfd64378c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:41.163405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.165068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.185029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e7c88-0c19-4b82-af5e-3e563fce8b0a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.190342 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.190858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.191232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.191513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.192245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.193908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76c06d-c106-4fef-848b-8d0534ea9274 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.207770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1001/1992] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.209139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.209427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.209704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.217080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:41.230129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:41.242317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.253878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1002/1993] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:41.254953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.256737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.262811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d539-e9a9-4e7a-a94a-81321c3dc086 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.270894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6e8a2-c18e-41ef-b9e3-33b68f6efe18 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.276040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.276724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.277123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.277339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.278314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.279996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5beffdc-a0b7-4e63-9ffb-eb608154bd41 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.280956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.281654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.281965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.282250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.282961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.284951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.291682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1003/1994] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.292967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.293533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.293641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.307867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 992/1995] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/users => generated 310 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:41.308166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.315835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:41.317107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.319402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90892f7d-0976-4c72-a04e-0fccb6ae74ed admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.324270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.325211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.325505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.325616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1004/1996] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:41.326102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] PATH_INFO: `/v3/projects/308592e1c7e4432f833ee0e110ba0cc4/users/63a9d6f7e9eb471cbbc82af74a3c5978/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.326681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.336890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c7436-c4b2-4a17-9644-6b9664a15435 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.342161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.342576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=308592e1c7e4432f833ee0e110ba0cc4, user_id=63a9d6f7e9eb471cbbc82af74a3c5978, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.342874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.343170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.343449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.344314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.345254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.345965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.357103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1005/1997] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.357103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.360569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 993/1998] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:41 2022] PUT /identity/v3/projects/308592e1c7e4432f833ee0e110ba0cc4/users/63a9d6f7e9eb471cbbc82af74a3c5978/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:41.361364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.367590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83822d6-be4b-41c0-b92f-95bf782031aa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.424995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.425713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.426028 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.426304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.427024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:41.427334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.428356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455008c9-99e0-48e3-81cc-2993f2f42f20 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.428943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef9626-e02a-4a87-9c64-7ff82d5c2ef9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.433093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.433973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.434279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.434553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] PATH_INFO: `/v3/projects/3f870c4d861c471eb4b23f22f110700c/users/82cc4da03130429797515889185c9dc3/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.440314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1006/1999] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:41.442126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.442421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.442705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.454718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f870c4d861c471eb4b23f22f110700c, user_id=82cc4da03130429797515889185c9dc3, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.458104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.468930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] MFA Rules not processed for user `63a9d6f7e9eb471cbbc82af74a3c5978`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:41.470153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.471425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 994/2000] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:41 2022] PUT /identity/v3/projects/3f870c4d861c471eb4b23f22f110700c/users/82cc4da03130429797515889185c9dc3/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:41.472002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.481214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:41.537358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60645a29-096e-41f1-843f-961d88885688 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.539808 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1007/2001] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:41.540409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.542531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.543196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.543484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.543756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.544459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.546470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.554784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e8f0-9b35-4033-9b21-8d6426d91335 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.565664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.566968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.567189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:41.567301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.567561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.568399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.570191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.576792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 995/2002] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:41.577778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.579515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.586747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1008/2003] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.586747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.594332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b54f32-16d5-4c7b-baa3-15166958f331 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.639919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e8186-dda4-4bcb-9f78-ac2a8555cbea admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.643913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.645516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.645876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.646246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.654631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.655302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.655613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.655888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.656604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.658583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.680029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1009/2004] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:41.681087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.691973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd23e6-5cb3-43a1-a885-e9dc3205157f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.697677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.698749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.699121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.699505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] PATH_INFO: `/v3/projects/3f870c4d861c471eb4b23f22f110700c/users/82cc4da03130429797515889185c9dc3/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.702253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f870c4d861c471eb4b23f22f110700c, user_id=82cc4da03130429797515889185c9dc3, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.705432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.709682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.711320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a75909-8bdf-4e8f-89c2-232257b04e7e service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.718010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1010/2005] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:41 2022] PUT /identity/v3/projects/3f870c4d861c471eb4b23f22f110700c/users/82cc4da03130429797515889185c9dc3/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:41.718823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.720986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 996/2006] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:41.728728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.729203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.729572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.756364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] MFA Rules not processed for user `82cc4da03130429797515889185c9dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:41.757699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.781629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ead34-2372-42c0-8f89-f36c489b0770 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.786430 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.787099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.787390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.787669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.788395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.790033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.800409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1011/2007] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.801588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.811560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44122c17-4dab-43b3-89f4-2ed7f206ac94 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.816885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.817857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.818182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.818619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] PATH_INFO: `/v3/users/63a9d6f7e9eb471cbbc82af74a3c5978` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.819819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a9d6f7e9eb471cbbc82af74a3c5978)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.821727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.826416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:41.828365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 997/2008] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:41 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:41.829360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.839992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b718f5-b651-44e3-b565-5152bb72867b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:41.844702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:41.845395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:41.845683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:41.845978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:41.846728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:41.848392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:41.861006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 998/2009] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:41.861855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.864875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.887201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a9d6f7e9eb471cbbc82af74a3c5978'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.894830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a9d6f7e9eb471cbbc82af74a3c5978'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.895837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a9d6f7e9eb471cbbc82af74a3c5978'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.896792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a9d6f7e9eb471cbbc82af74a3c5978'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.901021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:41.905716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a9d6f7e9eb471cbbc82af74a3c5978'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:41.926549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1012/2010] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:41 2022] DELETE /identity/v3/users/63a9d6f7e9eb471cbbc82af74a3c5978 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:41.930797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:41.932334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:41.933629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0f1fc3-3493-43c7-b44d-cfe09290eba9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.000287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.000948 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.001267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.001542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.004319 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.024956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2c29d-1d1b-490b-a3ec-37edda053207 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.032874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.033763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.034090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.034352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] PATH_INFO: `/v3/projects/23f7430ff8b4440d83ca5e8c95a650f9/users/4f97a8d3381544a887f23b148d67131e/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.055836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f7430ff8b4440d83ca5e8c95a650f9, user_id=4f97a8d3381544a887f23b148d67131e, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.060263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5c154-9599-4656-b801-76857b36d478 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.075593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1013/2011] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:41 2022] PUT /identity/v3/projects/23f7430ff8b4440d83ca5e8c95a650f9/users/4f97a8d3381544a887f23b148d67131e/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:42.077914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.078424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.078880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.080161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.081684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da6aa8-b58a-4acf-aeba-3d87a8a4a016 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.096629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 999/2012] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:41 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:42.098055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:42.098196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.098498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.098782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.099317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.135683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] MFA Rules not processed for user `4f97a8d3381544a887f23b148d67131e`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:42.137982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.198967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1014/2013] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:42.217904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.219423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.230024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1000/2014] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:42.230405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.235178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:42.255185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0638a-676f-4f20-a4f6-164cfb8d5202 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.267889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.268571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.268914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.269284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.270291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.272453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.306636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:42.321276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1001/2015] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/projects => generated 391 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:42.325774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.328835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dcc3d9-1d24-4b48-b976-d5507b90f93a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.411129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.411931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.412053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a280e1-8489-414b-b2fc-83091f792c6e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.412398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.412791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.416940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.417614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.417907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.418234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.418937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.421130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.423107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.424591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.435964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1015/2016] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:42.447534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1002/2017] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/users => generated 302 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:42.447665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.448492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.449531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.463659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e20580-71ce-4890-96ab-44c618c3e5bd admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.489041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.489848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.490185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.490439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.491530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.493231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.508968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1003/2018] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:42.509934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.512604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.535885 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d6eb-e597-4caa-9cd4-c54120680f22 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.540098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.540870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.541190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.541580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.544474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.579138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1ea86-b280-4ca9-8a93-ac5213f190a8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.584600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.585259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.585549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.586118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.587897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.622548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.624183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.634465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1016/2019] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:42.635292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.647517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956f0e1-ba7a-41b2-ad1e-c8e849e253c5 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.653449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.654594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.654958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.655386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] PATH_INFO: `/v3/projects/118e97a6830048cdac9baf39b6dc5fe8/users/c432510b32bd4bbcb9dcab60b1f842cc/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.672080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.673227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=118e97a6830048cdac9baf39b6dc5fe8, user_id=c432510b32bd4bbcb9dcab60b1f842cc, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.673884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.676371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.680977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:42.688675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1004/2020] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:42.703911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1017/2021] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:42 2022] PUT /identity/v3/projects/118e97a6830048cdac9baf39b6dc5fe8/users/c432510b32bd4bbcb9dcab60b1f842cc/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:42.708887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.721189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aee978-750d-4fa5-bd4f-14564c96bb69 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.746599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.748945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.766006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:42.818913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.819858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.820175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.820480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.822305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.826540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.839313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28826218-a049-45b0-9cdc-b4ecf69327fa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.841585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1005/2022] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:42.844254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.845253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.845597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.845877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] PATH_INFO: `/v3/projects/308592e1c7e4432f833ee0e110ba0cc4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.846010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:42.847273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=308592e1c7e4432f833ee0e110ba0cc4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.849223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.864745 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b012-63d3-40c6-8bc8-39ddf51eea89 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:42.871199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:42.872368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.872621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.873015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] PATH_INFO: `/v3/projects/118e97a6830048cdac9baf39b6dc5fe8/users/c432510b32bd4bbcb9dcab60b1f842cc/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.875605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118e97a6830048cdac9baf39b6dc5fe8, user_id=c432510b32bd4bbcb9dcab60b1f842cc, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:42.878344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62e71a-c2c7-4485-882d-1e6009518fb6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:42.892434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1006/2023] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:42 2022] PUT /identity/v3/projects/118e97a6830048cdac9baf39b6dc5fe8/users/c432510b32bd4bbcb9dcab60b1f842cc/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:42.904338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.904770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.905306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:42.922694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '308592e1c7e4432f833ee0e110ba0cc4', 'request_id': 'req-21decbd7-f360-4d18-8c9e-897d40f79201'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:42.932694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '308592e1c7e4432f833ee0e110ba0cc4', 'request_id': 'req-21decbd7-f360-4d18-8c9e-897d40f79201'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:42.942314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] MFA Rules not processed for user `c432510b32bd4bbcb9dcab60b1f842cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:42.943047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '308592e1c7e4432f833ee0e110ba0cc4', 'request_id': 'req-21decbd7-f360-4d18-8c9e-897d40f79201'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:42.943578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:42.960235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] The token cache is being invalidate because project 308592e1c7e4432f833ee0e110ba0cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:42.960530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:42.961403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21decbd7-f360-4d18-8c9e-897d40f79201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:42.963166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1018/2024] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:42 2022] DELETE /identity/v3/projects/308592e1c7e4432f833ee0e110ba0cc4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:42.968368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2de8-d7cb-4cc1-9efa-7211a53a686b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:42.968560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2de8-d7cb-4cc1-9efa-7211a53a686b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:42.968855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2de8-d7cb-4cc1-9efa-7211a53a686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.015383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5ccd2de8-d7cb-4cc1-9efa-7211a53a686b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:43.018588 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5ccd2de8-d7cb-4cc1-9efa-7211a53a686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.077811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1007/2025] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:43.090351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.091731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.121391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1019/2026] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:43.129230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656940-81c5-4e66-ab01-30270e233661 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.129709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656940-81c5-4e66-ab01-30270e233661 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.130029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db656940-81c5-4e66-ab01-30270e233661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.150375 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-db656940-81c5-4e66-ab01-30270e233661 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:43.151600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-db656940-81c5-4e66-ab01-30270e233661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.162755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1020/2027] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:43.165588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eec7e-dd9e-46fa-be08-91fa6e16bb24 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.166853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db656940-81c5-4e66-ab01-30270e233661 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.171462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.171843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.172414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.172741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.180394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db656940-81c5-4e66-ab01-30270e233661 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.185651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.186360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.186652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.186939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.187060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.187674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:43.188106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.188892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.189886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.204277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1021/2028] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:52:43.205111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1008/2029] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:43.209044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.221786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92434ab-7794-424a-9869-e7b401d4f4b2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.228502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.229227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.229544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.229895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.230758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.232716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.236380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:43.269442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1022/2030] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:43 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:43.271270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.274994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.283426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228ad1b-55bd-401a-9214-3e8d352a99fe service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.341634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c9112-0445-4052-98a8-6004343b7a90 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.345874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.346663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.347166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.348120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.349106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:43.350745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.366715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.367463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.367813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.368194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.368918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.371087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.401734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1009/2031] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:43 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:43.402654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.404495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.418852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844ec7e-2212-4cb8-bfba-ab17d7a5f4e5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.423391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.424172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.424752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.424905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.426897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.427017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.428480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.438327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1023/2032] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:43.439241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.449935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62d18c-32d2-4078-9271-3bc59818a0c3 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.456252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.457558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.457902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.458331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.459124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.460803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.472129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1024/2033] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:43.476612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.488167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3442-6591-40ba-9bb6-054466bdeb14 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.495787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.497593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.498252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.498846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.500360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.502259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.502807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.504814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35497fd5-9e6e-4fab-995c-53a8e382efed service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.516702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1025/2034] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:43.516904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1010/2035] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:43.518209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.530636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac590239-aebb-42e8-bff5-0005625c9e03 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.537324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.538456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.538854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.539230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] PATH_INFO: `/v3/projects/4b826b796594404eaf27663561090fbf/users/ea4f93ca368843c19f307a5119303c96/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.563562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b826b796594404eaf27663561090fbf, user_id=ea4f93ca368843c19f307a5119303c96, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.567472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.592952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1026/2036] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:43 2022] PUT /identity/v3/projects/4b826b796594404eaf27663561090fbf/users/ea4f93ca368843c19f307a5119303c96/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:43.605678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.606254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.606254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.642463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] MFA Rules not processed for user `ea4f93ca368843c19f307a5119303c96`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:43.643884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:43.719987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1011/2037] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:43.731121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:43.807104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70433bc-706b-4921-bc2d-5056177fed02 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:43.812039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:43.812676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:43.812973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:43.813234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:43.821052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:43.824243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:43.826468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:43.837989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1027/2038] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:48.921432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:48.923075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:48.940252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:48.996422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:49.001337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232bd19-9c01-4e94-8a4c-e9bf3558dc16 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.009535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.010447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.010734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.010846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e29e-5dd3-45c6-9a55-4e1a14c93f4a service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.011696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] PATH_INFO: `/v3/users/4f97a8d3381544a887f23b148d67131e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.012472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f97a8d3381544a887f23b148d67131e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.014847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.015512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.016414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.016699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.016982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] PATH_INFO: `/v3/users/ea4f93ca368843c19f307a5119303c96` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.018271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4f93ca368843c19f307a5119303c96)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.020480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.067805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f97a8d3381544a887f23b148d67131e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.077689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4f93ca368843c19f307a5119303c96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.081269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f97a8d3381544a887f23b148d67131e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.082151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f97a8d3381544a887f23b148d67131e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.082824 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f97a8d3381544a887f23b148d67131e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.089835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4f93ca368843c19f307a5119303c96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.090512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4f93ca368843c19f307a5119303c96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.091158 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4f93ca368843c19f307a5119303c96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.092622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-339b239a-39ab-441b-9f15-b65fbdcc1cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f97a8d3381544a887f23b148d67131e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.100662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e42cf194-6cd4-4e58-b852-74a313340c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4f93ca368843c19f307a5119303c96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.110060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1012/2039] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:48 2022] DELETE /identity/v3/users/4f97a8d3381544a887f23b148d67131e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:49.112987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.113274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.113567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.124082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1028/2040] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:48 2022] DELETE /identity/v3/users/ea4f93ca368843c19f307a5119303c96 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:49.128550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.128840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.129126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.131582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:49.132789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.149484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:49.150790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.200017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1013/2041] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:49.210771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.212352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.218537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1029/2042] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:49.228767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.230649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.257904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:49.280631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad1afc-3fe7-469b-9831-e6efd4af2c56 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.284710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.285501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.285861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.286782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.298118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.298688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.308408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891a493-c665-4a31-8cb4-2af30df5dd7a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.309512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1014/2043] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:49.317334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.318236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.318522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.318796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.328525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.329972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.341417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1030/2044] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:49.590564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.593125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.612651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4192-3c5b-4ca3-a453-98059f9546e3 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.618913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.620113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.620777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.620890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] PATH_INFO: `/v3/users/7e974e9a7cbe420e960f290086cc75c7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.623095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e974e9a7cbe420e960f290086cc75c7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.625682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.645166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:49.688242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.689984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.694972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e974e9a7cbe420e960f290086cc75c7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.709714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35783d62-c2c6-40fa-af58-561cbd4ce800 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.710887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e974e9a7cbe420e960f290086cc75c7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.711683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e974e9a7cbe420e960f290086cc75c7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.712991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e974e9a7cbe420e960f290086cc75c7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.715968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.716949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.717302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.717842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] PATH_INFO: `/v3/projects/23f7430ff8b4440d83ca5e8c95a650f9` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.719772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f7430ff8b4440d83ca5e8c95a650f9)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.721818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.731271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e974e9a7cbe420e960f290086cc75c7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.760676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1015/2045] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:49 2022] DELETE /identity/v3/users/7e974e9a7cbe420e960f290086cc75c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:49.766884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.768397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.789303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f7430ff8b4440d83ca5e8c95a650f9', 'request_id': 'req-8f602327-a569-43cf-bc21-67d88e106725'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.799880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f7430ff8b4440d83ca5e8c95a650f9', 'request_id': 'req-8f602327-a569-43cf-bc21-67d88e106725'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.808703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f7430ff8b4440d83ca5e8c95a650f9', 'request_id': 'req-8f602327-a569-43cf-bc21-67d88e106725'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.821557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] The token cache is being invalidate because project 23f7430ff8b4440d83ca5e8c95a650f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:49.821851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.822578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-8f602327-a569-43cf-bc21-67d88e106725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:49.827347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1031/2046] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:49 2022] DELETE /identity/v3/projects/23f7430ff8b4440d83ca5e8c95a650f9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:49.827347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.827347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.827347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.846932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:49.848284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.859791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f52ce2-77da-4af2-b574-6fe174044075 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.935466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1032/2047] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:49.939513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:49.941034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.941519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:49.941917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.942914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.943365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] PATH_INFO: `/v3/projects/4b826b796594404eaf27663561090fbf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.944664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b826b796594404eaf27663561090fbf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.946735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.954503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8eaaf-19f6-4c6b-9f36-ccffa08053a3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:49.960984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:49.961977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:49.962314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:49.962647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] PATH_INFO: `/v3/users/c432510b32bd4bbcb9dcab60b1f842cc` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:49.964068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c432510b32bd4bbcb9dcab60b1f842cc)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:49.966188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:49.995613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.002992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b826b796594404eaf27663561090fbf', 'request_id': 'req-53d6f503-35a8-499c-bcdc-58a87d6e45a8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.011621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b826b796594404eaf27663561090fbf', 'request_id': 'req-53d6f503-35a8-499c-bcdc-58a87d6e45a8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.014159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.018116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b826b796594404eaf27663561090fbf', 'request_id': 'req-53d6f503-35a8-499c-bcdc-58a87d6e45a8'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.025557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c432510b32bd4bbcb9dcab60b1f842cc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.031744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] The token cache is being invalidate because project 4b826b796594404eaf27663561090fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:50.032139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.032543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c432510b32bd4bbcb9dcab60b1f842cc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.032837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.033387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c432510b32bd4bbcb9dcab60b1f842cc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.034112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c432510b32bd4bbcb9dcab60b1f842cc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.034449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1016/2048] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:49 2022] DELETE /identity/v3/projects/4b826b796594404eaf27663561090fbf => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:50.035254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.036553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.045359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cc07d6f2-7da6-461b-9d07-5124643bc766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c432510b32bd4bbcb9dcab60b1f842cc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.065447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1033/2049] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:49 2022] DELETE /identity/v3/users/c432510b32bd4bbcb9dcab60b1f842cc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:50.067065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d14d-e5ad-47c9-b8d8-4efaf8560cbb None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.068148 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d14d-e5ad-47c9-b8d8-4efaf8560cbb None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.068570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d14d-e5ad-47c9-b8d8-4efaf8560cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.089017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9527d14d-e5ad-47c9-b8d8-4efaf8560cbb None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:50.090520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9527d14d-e5ad-47c9-b8d8-4efaf8560cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.097502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6f503-35a8-499c-bcdc-58a87d6e45a8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.161649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.162465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.162764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.163044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.164901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.168881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1034/2050] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:50.178275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.178661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.179004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.180931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.183182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.198398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1017/2051] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:50.200590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:50.201821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.210830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1035/2052] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:50.215825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.227884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e21f05-4845-4c4d-8e0e-f6c2cb4b1c86 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.251557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.252723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.252723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.253200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.254216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:50.255668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.257980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.275289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1018/2053] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:50.276773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.286833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2da462-1d8f-4dc0-9f38-5a1fe61497fd None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.293698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.294759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.295298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.295816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.296768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.298898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.330136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1036/2054] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:50.334224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.345314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbce85-3f15-422c-ad10-e96863b8eef4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.406828 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.408671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.432239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.432850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.433283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.433687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.434441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.436868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.467141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1019/2055] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:50.470536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.483638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.490105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4980bc1-ef33-4370-b683-04cd7aecdf15 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.497844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.499106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.499787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.500299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.501865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.503887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.523713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c5b11-e061-435d-8635-731a3e77abce admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.531115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1020/2056] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:50.532490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.532809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.533587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.533881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.534321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] PATH_INFO: `/v3/projects/118e97a6830048cdac9baf39b6dc5fe8` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.535577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] RBAC: Authorizing `identity:delete_project(project_id=118e97a6830048cdac9baf39b6dc5fe8)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.538170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.544563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df543b8e-2ec4-439f-ac57-7736ef7d30fe admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.552203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.553756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.554096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.554789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.555533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.557252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.581086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1021/2057] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:50.582449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.586063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.594274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.599386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5f869-9f56-4f69-b198-b56523592a42 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.618629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118e97a6830048cdac9baf39b6dc5fe8', 'request_id': 'req-4c352dea-e3e3-4900-8c17-089fe3dfc46d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.627716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118e97a6830048cdac9baf39b6dc5fe8', 'request_id': 'req-4c352dea-e3e3-4900-8c17-089fe3dfc46d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.635841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118e97a6830048cdac9baf39b6dc5fe8', 'request_id': 'req-4c352dea-e3e3-4900-8c17-089fe3dfc46d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.648335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] The token cache is being invalidate because project 118e97a6830048cdac9baf39b6dc5fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:50.648707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.649457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.651171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1037/2058] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:50 2022] DELETE /identity/v3/projects/118e97a6830048cdac9baf39b6dc5fe8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:50.652060 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.653415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.667507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.668599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.669241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.669241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] PATH_INFO: `/v3/projects/ae263982ccbe4cecb046ae63889618e3` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.670410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae263982ccbe4cecb046ae63889618e3)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.672874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.727246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae263982ccbe4cecb046ae63889618e3', 'request_id': 'req-67ae50cf-e141-45a5-bbd3-0627525d35e7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.727396 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c352dea-e3e3-4900-8c17-089fe3dfc46d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.733759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae263982ccbe4cecb046ae63889618e3', 'request_id': 'req-67ae50cf-e141-45a5-bbd3-0627525d35e7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.740676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae263982ccbe4cecb046ae63889618e3', 'request_id': 'req-67ae50cf-e141-45a5-bbd3-0627525d35e7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.758643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] The token cache is being invalidate because project ae263982ccbe4cecb046ae63889618e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:50.758948 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.760187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ae50cf-e141-45a5-bbd3-0627525d35e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.764567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1022/2059] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:50 2022] DELETE /identity/v3/projects/ae263982ccbe4cecb046ae63889618e3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:50.767888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315977b5-d664-42a2-a0ca-3728a266cd02 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.768179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315977b5-d664-42a2-a0ca-3728a266cd02 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.768462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315977b5-d664-42a2-a0ca-3728a266cd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.789692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-315977b5-d664-42a2-a0ca-3728a266cd02 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:50.792465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-315977b5-d664-42a2-a0ca-3728a266cd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.797707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.798718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.799095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.799448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] PATH_INFO: `/v3/projects/740a3934b7a2462fa458231439b4a3aa/users/f16693cdac034450bc435f645ca4be58/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.818678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740a3934b7a2462fa458231439b4a3aa, user_id=f16693cdac034450bc435f645ca4be58, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.821847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.837719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1038/2060] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:50 2022] PUT /identity/v3/projects/740a3934b7a2462fa458231439b4a3aa/users/f16693cdac034450bc435f645ca4be58/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:50.838367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:50.839991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.881017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1023/2061] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:50.882943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057bd1-806d-43e0-ad3f-e2964014cb51 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.883237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057bd1-806d-43e0-ad3f-e2964014cb51 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.883514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057bd1-806d-43e0-ad3f-e2964014cb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.897863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.908786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:50.913381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-55057bd1-806d-43e0-ad3f-e2964014cb51 None None] MFA Rules not processed for user `f16693cdac034450bc435f645ca4be58`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:50.914942 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-55057bd1-806d-43e0-ad3f-e2964014cb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:50.921117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7870b-b0d9-4a4a-a618-655d63c90377 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:50.926489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:50.927949 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:50.928230 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:50.928506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] PATH_INFO: `/v3/users/457626fe83264f0a99d242c739879182` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:50.929794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=457626fe83264f0a99d242c739879182)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:50.931806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:50.983620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457626fe83264f0a99d242c739879182'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.995018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457626fe83264f0a99d242c739879182'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:50.999946 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1024/2062] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:51.000420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457626fe83264f0a99d242c739879182'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.000503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.000503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.000503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.002352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457626fe83264f0a99d242c739879182'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.018541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457626fe83264f0a99d242c739879182'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.032037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:51.033403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.046104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1025/2063] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:51.048982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1039/2064] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:50 2022] DELETE /identity/v3/users/457626fe83264f0a99d242c739879182 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:51.049572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.050977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.051376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.145245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b50a6d-dc09-4120-b4cc-438c43251283 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.153568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69cb08-258d-4a39-b1ea-d5fe5f2ef3c3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.163398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.164224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.164624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.164908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.166159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:51.166657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.170887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.199563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1040/2065] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:51.201749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.211650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.212427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.212912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.213317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.215566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.225855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44861b-101a-4349-979e-03bd1f2ce856 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.232502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.233382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.233931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.234731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.235596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.238741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.272411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1041/2066] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:51 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:51.273294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.285177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.316694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.318658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.330776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1026/2067] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:51.338341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.341056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.342081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:51.366690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553715d-9e5f-4022-936a-e520a6dcd6c6 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.373268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.374965 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.375948 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.376351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] PATH_INFO: `/v3/users/993f5e0b1a824b2fb43d161c4d3b5465` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.377906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993f5e0b1a824b2fb43d161c4d3b5465)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.378818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5b2a1-9616-48da-8aa3-a6e04c5553a7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.381194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.383489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:51.385536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.386649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.386946 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.387232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] PATH_INFO: `/v3/users/4b61e9897f8e4c8ab2fd8cd6b5244ca5` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.388739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b61e9897f8e4c8ab2fd8cd6b5244ca5)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.390904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.459338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993f5e0b1a824b2fb43d161c4d3b5465'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.467535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b61e9897f8e4c8ab2fd8cd6b5244ca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.476704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b61e9897f8e4c8ab2fd8cd6b5244ca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.477463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b61e9897f8e4c8ab2fd8cd6b5244ca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.481071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b61e9897f8e4c8ab2fd8cd6b5244ca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.482492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993f5e0b1a824b2fb43d161c4d3b5465'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.483679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993f5e0b1a824b2fb43d161c4d3b5465'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.485094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993f5e0b1a824b2fb43d161c4d3b5465'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.491780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b61e9897f8e4c8ab2fd8cd6b5244ca5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.505978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993f5e0b1a824b2fb43d161c4d3b5465'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.516679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1042/2068] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:51 2022] DELETE /identity/v3/users/4b61e9897f8e4c8ab2fd8cd6b5244ca5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:51.518219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.522129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.531247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1027/2069] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:51 2022] DELETE /identity/v3/users/993f5e0b1a824b2fb43d161c4d3b5465 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:51.536052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.540069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.613880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6419e0-faf4-4056-83c9-b03686530f30 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.650595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db563c-a6e2-41a3-aa2a-770fb5e94c25 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.657338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.658252 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.658638 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.658928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] PATH_INFO: `/v3/projects/3bdacb98af9e40f9a29532b4562a23ee` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.660190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bdacb98af9e40f9a29532b4562a23ee)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.663289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.692113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.693056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.693525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.694652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.695417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.697903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313671a-8571-4a63-8471-65d4ce90502d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.714628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:51.731009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1043/2070] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:51 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:51.734484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bdacb98af9e40f9a29532b4562a23ee', 'request_id': 'req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.740474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.740474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.740474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.743135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bdacb98af9e40f9a29532b4562a23ee', 'request_id': 'req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.750276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bdacb98af9e40f9a29532b4562a23ee', 'request_id': 'req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.773109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] The token cache is being invalidate because project 3bdacb98af9e40f9a29532b4562a23ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:51.773481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.774147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.778444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1028/2071] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:51 2022] DELETE /identity/v3/projects/3bdacb98af9e40f9a29532b4562a23ee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:51.780118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.781855 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.785166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:51.787554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.795165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:51.866971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb7e9f-6227-4d90-bac5-2d650f2c8e0f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.870364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1044/2072] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:51.871437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.872995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.874704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.875694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.876094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.876475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] PATH_INFO: `/v3/projects/e1d35cdfa73449c2bd040f966d9f27a2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.878615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d35cdfa73449c2bd040f966d9f27a2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.881690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.886200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287b22-fd13-4db2-becb-024adceebfab None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.891417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.892083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.892371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.892646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.893379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.895288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:51.907390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1045/2073] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:51.908906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.911861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.927786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ec361-4aee-4471-8f56-2b97840f9bf2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:51.936321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d35cdfa73449c2bd040f966d9f27a2', 'request_id': 'req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.943480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d35cdfa73449c2bd040f966d9f27a2', 'request_id': 'req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.949246 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d35cdfa73449c2bd040f966d9f27a2', 'request_id': 'req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.963529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] The token cache is being invalidate because project e1d35cdfa73449c2bd040f966d9f27a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:51.963823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.964613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:51.966426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1029/2074] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:51 2022] DELETE /identity/v3/projects/e1d35cdfa73449c2bd040f966d9f27a2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:51.967210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:51.968524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:51.993172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:51.994001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:51.994291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:51.994565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] PATH_INFO: `/v3/users/43f776434ae540b190e5ef531abbc289` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:51.996058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43f776434ae540b190e5ef531abbc289)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:51.998093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.031430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9275f6b-3134-4718-b0c0-e8f2cc2c77b7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.035371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.036034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.036340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.036691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.038531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.046997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43f776434ae540b190e5ef531abbc289'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.050652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.052124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.055083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43f776434ae540b190e5ef531abbc289'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.055853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43f776434ae540b190e5ef531abbc289'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.056477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43f776434ae540b190e5ef531abbc289'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.062125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1030/2075] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:52.062891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.063935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-209687ff-96c3-4887-867b-767ab3705df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43f776434ae540b190e5ef531abbc289'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.064232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.076775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963ccdc-254b-455a-a98a-edc6a1190a11 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.082177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.082720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.083576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.083882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.084667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1046/2076] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:51 2022] DELETE /identity/v3/users/43f776434ae540b190e5ef531abbc289 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:52.085915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9f9-34ec-4fd8-8a99-a27b96065e7c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.086243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9f9-34ec-4fd8-8a99-a27b96065e7c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.086552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8a9f9-34ec-4fd8-8a99-a27b96065e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.087377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.089043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.094258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:52.107839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1031/2077] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:52.113373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.113864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-80b8a9f9-34ec-4fd8-8a99-a27b96065e7c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:52.116720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.116867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-80b8a9f9-34ec-4fd8-8a99-a27b96065e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.134521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c763b455-4524-4b6e-8730-ef954197a03f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.180809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:52.192789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1047/2078] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:52.201775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.202175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.202713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.223313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.225439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.225723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:52.225861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.226226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] PATH_INFO: `/v3/projects/df5b29d47d554c588abfc86dfbb83b58/users/1db5afe984724a74b6dfe0960c9ce858/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.227362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.249645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5b29d47d554c588abfc86dfbb83b58, user_id=1db5afe984724a74b6dfe0960c9ce858, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.253033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331241c-d720-4c91-b2fb-f708582d1ec4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.255112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1048/2079] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:52.256345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.266422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa4357-b06d-47a6-aaa1-2e0f132ca1e2 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.277203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1032/2080] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:52 2022] PUT /identity/v3/projects/df5b29d47d554c588abfc86dfbb83b58/users/1db5afe984724a74b6dfe0960c9ce858/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:52.288375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.288656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.289073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.320028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] MFA Rules not processed for user `1db5afe984724a74b6dfe0960c9ce858`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:52.322172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.346770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.347439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.347720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.347996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.349310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:52.349939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.351315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.368114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1049/2081] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:52.369814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.382692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0dd4b-2ab0-4efe-9951-483cce9f2097 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.388044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.388821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.388943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.389286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.390089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.392132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.407695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1033/2082] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:52.413428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.423642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1050/2083] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:52.425758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef617627-cd00-4e32-a500-051673ab4de8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.427827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.499535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.499535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.499535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.499535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.500068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.501491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.514080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291119d7-b411-4f84-815d-1ffd6caca27b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.520996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.521995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.522437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.522876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.523591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.525558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.543560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:52.545875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1034/2084] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:52.550331 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.552466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.553388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1051/2085] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:52.554265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.556858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.570888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe61c4-807c-4b0d-9d14-94b5943725d9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.589054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:52.624079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a2a615-b3b8-4841-bbe2-7182fef2b318 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.628974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.630371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.630775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.630998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.631111 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] PATH_INFO: `/v3/projects/e70612188b15468ca8171f28028e79a0` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.631871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.632259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70612188b15468ca8171f28028e79a0)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.632411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.632630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] PATH_INFO: `/v3/projects/d7470c61398142bfad03a4302d950ec6` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.633809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7470c61398142bfad03a4302d950ec6)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.634260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.635775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.683241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70612188b15468ca8171f28028e79a0', 'request_id': 'req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.683732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7470c61398142bfad03a4302d950ec6', 'request_id': 'req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.689398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70612188b15468ca8171f28028e79a0', 'request_id': 'req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.694358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7470c61398142bfad03a4302d950ec6', 'request_id': 'req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.695782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70612188b15468ca8171f28028e79a0', 'request_id': 'req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.701203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7470c61398142bfad03a4302d950ec6', 'request_id': 'req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.709770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] The token cache is being invalidate because project e70612188b15468ca8171f28028e79a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:52.710205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.710945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.712617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1035/2086] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:52 2022] DELETE /identity/v3/projects/e70612188b15468ca8171f28028e79a0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:52.713309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] The token cache is being invalidate because project d7470c61398142bfad03a4302d950ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:52.713589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.713814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.714457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.715797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.716388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1052/2087] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:52 2022] DELETE /identity/v3/projects/d7470c61398142bfad03a4302d950ec6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:52.717229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.718542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.781037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373383f8-79e3-4e8e-a1ba-d1689b9f6e5a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.782503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afae6cf-8f7f-47a0-a71b-a1b547862ea8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.785793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.786513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.786797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.787067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.787210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.787752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.787867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.788152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.788428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.789182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.789673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc926757-7e0d-48bb-8e48-3e096e877aad admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.790797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.800749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1053/2088] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:52.801665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.803516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.812358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1036/2089] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:52.813499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.813783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.814087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.820048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11760b0-07cd-44cd-af3a-6b06b1352168 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.824865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.825765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.826211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.826493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] PATH_INFO: `/v3/users/82cc4da03130429797515889185c9dc3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.827704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82cc4da03130429797515889185c9dc3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.829644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.834773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:52.835997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.853537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1037/2090] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:52.855007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.856134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:52.864971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc636f-4873-4dfa-bd07-1d22ee1243b3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.870691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.871255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.871542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.871878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.872904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.874271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.885666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1038/2091] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:52.886372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.889904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.892855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82cc4da03130429797515889185c9dc3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.901965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82cc4da03130429797515889185c9dc3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.902776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82cc4da03130429797515889185c9dc3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.903512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82cc4da03130429797515889185c9dc3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.904790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe479f-33b8-440a-9836-bf97d967be9b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.910227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:52.911182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.911521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.911715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.912396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:52.912855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-83b8a4f8-4eb1-4017-aef8-48c4e1d23da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82cc4da03130429797515889185c9dc3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:52.915262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:52.932070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1054/2092] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:52 2022] DELETE /identity/v3/users/82cc4da03130429797515889185c9dc3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:52.939892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1039/2093] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:52.939892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:52.940316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:52.940897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:52.941189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:52.943153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.960065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532587e8-f37f-42d5-8274-93d6994b116d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:52.962247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:52.963812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:52.986598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:53.038556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1055/2094] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:53.042763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.044269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.058298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad1bf-22fd-40ee-9f31-24f2d3628463 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.064824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.065900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.066256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.067220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] PATH_INFO: `/v3/projects/3568da27f7274c1ab661e60402201c33/users/30a3e07242f34edea8e3c1e6793043b7/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.067388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.068756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.069050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.069332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] PATH_INFO: `/v3/projects/33781de828264846b90b5cbbbeee7fe5/users/06f644c1c44d4dafacbd5bbdde323ff0/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.084445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568da27f7274c1ab661e60402201c33, user_id=30a3e07242f34edea8e3c1e6793043b7, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.087823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.091184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33781de828264846b90b5cbbbeee7fe5, user_id=06f644c1c44d4dafacbd5bbdde323ff0, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.094968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.110985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1056/2095] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:53 2022] PUT /identity/v3/projects/3568da27f7274c1ab661e60402201c33/users/30a3e07242f34edea8e3c1e6793043b7/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:53.111972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.118352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1040/2096] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:52 2022] PUT /identity/v3/projects/33781de828264846b90b5cbbbeee7fe5/users/06f644c1c44d4dafacbd5bbdde323ff0/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:53.119214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.214320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480d8ce-5a78-4242-9a90-9c31859c40a0 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.219367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.220038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.220327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.220605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.221342 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.223039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea98f23-30b8-46bf-8868-fbd240c2c900 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.254026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1041/2097] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:53.256182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.256636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.256927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.272500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef034ed9-5002-4fef-a371-e5e325ad6ad8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.277906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:53.279171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.292337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.294596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.294711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1042/2098] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:53.295189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.295630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.295902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.296841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:53.306807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.308615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.321399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113d05f-f468-4385-86bf-98984c8b4ae9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.342170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.342772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1057/2099] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:53.343229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.343634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.343750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.343843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] PATH_INFO: `/v3/projects/3568da27f7274c1ab661e60402201c33/users/30a3e07242f34edea8e3c1e6793043b7/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.345893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.346719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3568da27f7274c1ab661e60402201c33, user_id=30a3e07242f34edea8e3c1e6793043b7, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.349907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.363424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760e28c-eb8b-4a6f-80bf-42b0eb08261d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.366129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1043/2100] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:53 2022] PUT /identity/v3/projects/3568da27f7274c1ab661e60402201c33/users/30a3e07242f34edea8e3c1e6793043b7/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:53.370055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.370187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:53.371741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.431318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.432191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.432478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.432784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] PATH_INFO: `/v3/projects/3f870c4d861c471eb4b23f22f110700c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.434014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f870c4d861c471eb4b23f22f110700c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.434765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95733e3-ece8-4dd1-ad45-773284e00103 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.435957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.438494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.439165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.439451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.439729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.449511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.451016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.460886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1044/2101] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:53.461660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.471279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fefbe-36c7-4c39-8633-ddf641af1865 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.479216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.479882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.480160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.480447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.481119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.483094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b80c8-12c8-4722-8a15-efecde378d0b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.493999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f870c4d861c471eb4b23f22f110700c', 'request_id': 'req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:53.502575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f870c4d861c471eb4b23f22f110700c', 'request_id': 'req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:53.504727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:53.516323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1045/2102] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:53.516516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f870c4d861c471eb4b23f22f110700c', 'request_id': 'req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:53.516607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282aecb-04ce-4743-8daa-fc7d85b3c51a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.516607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282aecb-04ce-4743-8daa-fc7d85b3c51a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.516607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282aecb-04ce-4743-8daa-fc7d85b3c51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.537251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] The token cache is being invalidate because project 3f870c4d861c471eb4b23f22f110700c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:53.537944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:53.539202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:53.542076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1058/2103] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:53 2022] DELETE /identity/v3/projects/3f870c4d861c471eb4b23f22f110700c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:53.545701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c282aecb-04ce-4743-8daa-fc7d85b3c51a None None] MFA Rules not processed for user `30a3e07242f34edea8e3c1e6793043b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:53.547415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c282aecb-04ce-4743-8daa-fc7d85b3c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.548391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.550917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.627152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1046/2104] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:53.628610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.628957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.629579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.654360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:53.655649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.692590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbf2e-9bfa-4eb7-ba5e-9e3593db01d0 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.734410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1047/2105] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:53.735185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.736596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.772559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:53.797265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.797799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.798445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.798813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.799773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.802477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6030a-3159-4102-82fe-3a918226533b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.827596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e521de-de0d-4ea3-82b9-33e15b72d0bd None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.832665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1059/2106] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/users => generated 292 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:53.832665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.832665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.832665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.838727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.840686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.841920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.841920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.854661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:53.855600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:53.857542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.859634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.864557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1060/2107] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:53.865176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.877064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb78aae-5b5c-4675-9da2-ba3af020c475 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.884299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.885131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.885574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.885944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.887372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.888781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1048/2108] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:53.890230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.891427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.920397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac67547-1da4-463a-8333-339f0693df1c service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.924711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1061/2109] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:53.925744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.931653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.932907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.933539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.934555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.938871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:53.939451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.941906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.956860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1049/2110] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:53.960974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf55ab-efa5-4392-b848-7c3da2861084 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.965593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:53.978566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:53.979517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:53.980168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:53.980594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:53.981688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:53.984116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:53.984304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff14e9-da00-45e2-ba93-886005356202 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:53.999568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1062/2111] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:54.001613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.003407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.006753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.007056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.007398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.008140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.011028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.018254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33960e42-627e-445c-be1b-0fea8e3453bc admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.024717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.025632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.025917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.026256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] PATH_INFO: `/v3/projects/c9daf941830f4a31b8665d2737172b12/users/94854f6c71a640939db8510c01545dc5/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.033166 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:54.046627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9daf941830f4a31b8665d2737172b12, user_id=94854f6c71a640939db8510c01545dc5, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.049715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.052019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1050/2112] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:53 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:54.052821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.079058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1063/2113] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:54 2022] PUT /identity/v3/projects/c9daf941830f4a31b8665d2737172b12/users/94854f6c71a640939db8510c01545dc5/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:54.079058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.122096 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228b8d7-ad76-4c43-9d5b-32cea7378b33 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.152540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6a649-4874-4f09-8bfc-0096d9f2c9ac admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.157912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.158619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.158919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.159191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.159892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.161991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6030c9-a507-4227-b7c7-2550d95ff692 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.179774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.180570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.180962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.181306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.186666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1064/2114] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:54 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:54.186832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:54.186947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.186947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.186947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.217513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] MFA Rules not processed for user `94854f6c71a640939db8510c01545dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:54.219023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:54.265264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.267121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.279646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1051/2115] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:54.289829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.316372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442aba2-ecf3-4450-b2e7-675081a73000 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.316825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:54.325006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1065/2116] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:54.332804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.339378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.340965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:54.342484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.342806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.345134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.346068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.350451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.356537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef670c6-a674-481a-9c8b-b8f176aac112 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.361408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.362160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.362497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.362911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.378041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1052/2117] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:54.381749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.382620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.383259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.405126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460dfc1-00ae-4152-9568-2da41ec1e186 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.406712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1066/2118] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:54.412876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.414899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.417071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.417455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.421456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.424808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076eb109-56b1-49ca-8f3f-e533665a5ffe admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.451690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1053/2119] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:52:54.452238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.466555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055fff3-d02c-46eb-9a42-e13e3b18115c service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.479612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.480766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.481317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.481739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] PATH_INFO: `/v3/projects/86dbdab078974e978413d301d977a90f/users/ffa0e0592e564f4ea6b115b3535e94fd/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.518325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dbdab078974e978413d301d977a90f, user_id=ffa0e0592e564f4ea6b115b3535e94fd, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.524185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.552913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1067/2120] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:54 2022] PUT /identity/v3/projects/86dbdab078974e978413d301d977a90f/users/ffa0e0592e564f4ea6b115b3535e94fd/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:54.563035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.563452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.563828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.606353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] MFA Rules not processed for user `ffa0e0592e564f4ea6b115b3535e94fd`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:54.607593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:54.651139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:54.713375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1054/2121] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:54.714481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.878759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd11b8c-1f21-4b28-9f0b-fca95acedb3c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:54.896261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:54.897235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:54.897621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:54.898618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:54.901698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:54.905812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:54.964594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1068/2122] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:54 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:54.969916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:54.987090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0f9a4-bcff-48b1-95b7-29161c3e9fba None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.061842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.062669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.063053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.064153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.064986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.067744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.106406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1055/2123] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:54 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:52:55.107756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:55.135305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebd2b3-d2d5-48e8-a041-52abc9d80ea1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.142033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.142954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.143474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.144067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.145253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.147613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.168594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1069/2124] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:55.169484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:55.187255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84532cb-409f-4668-826b-33a77444625e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.194029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.195445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.195934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.196978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] PATH_INFO: `/v3/projects/0a199aa3bb954684a93f2e6a886b4eed/users/34cddeed3d834301876fbc5e982615db/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.217425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a199aa3bb954684a93f2e6a886b4eed, user_id=34cddeed3d834301876fbc5e982615db, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.224918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.256521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1056/2125] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:55 2022] PUT /identity/v3/projects/0a199aa3bb954684a93f2e6a886b4eed/users/34cddeed3d834301876fbc5e982615db/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:55.258758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:55.275919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8706967-0d64-4462-ab30-c5caf9345afb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.352156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.353361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.353361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.353678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.354401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.356330 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4713d7-6207-4d3b-bdab-a5e0d157f3b0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.384293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1070/2126] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:55.388389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:55.403832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5909e6-5224-4918-b64e-92d7583ea040 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.408762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.409669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.409975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.410757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] PATH_INFO: `/v3/projects/0a199aa3bb954684a93f2e6a886b4eed/users/34cddeed3d834301876fbc5e982615db/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.412919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a199aa3bb954684a93f2e6a886b4eed, user_id=34cddeed3d834301876fbc5e982615db, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.415652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.427484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1057/2127] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:55 2022] PUT /identity/v3/projects/0a199aa3bb954684a93f2e6a886b4eed/users/34cddeed3d834301876fbc5e982615db/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:55.438655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.438952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.439229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.469022 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] MFA Rules not processed for user `34cddeed3d834301876fbc5e982615db`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:55.470038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:55.493795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:55.556467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1071/2128] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:55.567879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:55.670049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c2a43-5456-497f-90a6-6ea186c0b3f3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:55.674053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:55.674717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:55.675005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:55.675284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:55.689527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:55.691265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab994ae9-71c8-4246-821f-9b7d04144391 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:55.703858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1058/2129] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:56.379392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:56.482139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57349d-fc57-47c7-9026-b8399ff2f6c7 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:56.500853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:56.502047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:56.502335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:56.502610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] PATH_INFO: `/v3/users/ffa0e0592e564f4ea6b115b3535e94fd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:56.512211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa0e0592e564f4ea6b115b3535e94fd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:56.514714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:56.625030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa0e0592e564f4ea6b115b3535e94fd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:56.641160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa0e0592e564f4ea6b115b3535e94fd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:56.641932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa0e0592e564f4ea6b115b3535e94fd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:56.646915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa0e0592e564f4ea6b115b3535e94fd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:56.671126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa0e0592e564f4ea6b115b3535e94fd'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:56.711625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1072/2130] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:56 2022] DELETE /identity/v3/users/ffa0e0592e564f4ea6b115b3535e94fd => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:56.715888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:56.716359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:56.716435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:56.752198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:56.753446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:56.826791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:56.914171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1059/2131] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:56.926268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:56.927193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:57.013006 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:57.038138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515d4e2-8219-43e5-a72c-60befba5c6df admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:57.043229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:57.043901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:57.044332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:57.044618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:57.059110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:57.060743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:57.105810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1073/2132] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:52:57.492071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:57.493854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:57.523007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044899c-6f92-4a04-a038-d2a71acb96c2 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:57.535253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:57.536258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:57.536653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:57.536804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] PATH_INFO: `/v3/users/94854f6c71a640939db8510c01545dc5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:57.538094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94854f6c71a640939db8510c01545dc5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:57.540039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:57.606195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:57.607686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:57.631068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92734f72-0848-48f8-bf34-769894e0e244 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:57.631641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94854f6c71a640939db8510c01545dc5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.637496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:57.639180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:57.640018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:57.640307 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] PATH_INFO: `/v3/projects/86dbdab078974e978413d301d977a90f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:57.643175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dbdab078974e978413d301d977a90f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:57.646628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:57.679332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94854f6c71a640939db8510c01545dc5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.684369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94854f6c71a640939db8510c01545dc5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.684369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94854f6c71a640939db8510c01545dc5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.698410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e98e4763-51b1-4568-8277-cc0a249fb57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94854f6c71a640939db8510c01545dc5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.728777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dbdab078974e978413d301d977a90f', 'request_id': 'req-86ac196e-199e-4a37-aa9e-c94d0c5999c2'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.730980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1060/2133] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:57 2022] DELETE /identity/v3/users/94854f6c71a640939db8510c01545dc5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:57.732525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:57.732722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:57.733004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:57.741346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dbdab078974e978413d301d977a90f', 'request_id': 'req-86ac196e-199e-4a37-aa9e-c94d0c5999c2'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.751298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dbdab078974e978413d301d977a90f', 'request_id': 'req-86ac196e-199e-4a37-aa9e-c94d0c5999c2'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.768352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] The token cache is being invalidate because project 86dbdab078974e978413d301d977a90f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:57.768761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.771152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:57.773835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1074/2134] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:57 2022] DELETE /identity/v3/projects/86dbdab078974e978413d301d977a90f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:57.774918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:57.778335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:57.779547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:57.781411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:57.949015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac196e-199e-4a37-aa9e-c94d0c5999c2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:57.970552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1061/2135] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:57.980719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:57.982702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:58.038334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:58.078060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf43c5-8c31-4db4-9c9b-b72273df4117 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:58.082178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:58.083152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:58.083560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:58.083904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:58.097266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:58.099534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:58.129361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:58.130170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1062/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:52:58.130899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:58.132327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:58.132623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] PATH_INFO: `/v3/users/34cddeed3d834301876fbc5e982615db` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:58.134928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=34cddeed3d834301876fbc5e982615db)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:58.140462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:58.181894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:58.282545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34cddeed3d834301876fbc5e982615db'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:58.310961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34cddeed3d834301876fbc5e982615db'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:58.311722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34cddeed3d834301876fbc5e982615db'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:58.312356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34cddeed3d834301876fbc5e982615db'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:58.342622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34cddeed3d834301876fbc5e982615db'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:58.390849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1075/2137] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:52:57 2022] DELETE /identity/v3/users/34cddeed3d834301876fbc5e982615db => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:52:58.908679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:58.932241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9543bce-02ff-44f7-8f40-168c029908df service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.043219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.045704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.104034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.106620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.107075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.107636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] PATH_INFO: `/v3/projects/0a199aa3bb954684a93f2e6a886b4eed` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.110165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a199aa3bb954684a93f2e6a886b4eed)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.116502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.213790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd8b02-33fb-4f4b-9cad-aa51c5b1e3bb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.231091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.232261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.232678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.233074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] PATH_INFO: `/v3/projects/c9daf941830f4a31b8665d2737172b12` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.234692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9daf941830f4a31b8665d2737172b12)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.240897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.292984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a199aa3bb954684a93f2e6a886b4eed', 'request_id': 'req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.304733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a199aa3bb954684a93f2e6a886b4eed', 'request_id': 'req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.318621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a199aa3bb954684a93f2e6a886b4eed', 'request_id': 'req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.341498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] The token cache is being invalidate because project 0a199aa3bb954684a93f2e6a886b4eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:59.341798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.343797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a3adbb63-f62c-4cdc-bf96-bdeea92fc065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.350854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1063/2138] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:58 2022] DELETE /identity/v3/projects/0a199aa3bb954684a93f2e6a886b4eed => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:59.351026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9daf941830f4a31b8665d2737172b12', 'request_id': 'req-51ed14fa-6375-482c-a281-7be6aa19abfc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.352609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0dd56-dd3a-48ed-b0cb-691637fe41d8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.352897 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0dd56-dd3a-48ed-b0cb-691637fe41d8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.353184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0dd56-dd3a-48ed-b0cb-691637fe41d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.367026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9daf941830f4a31b8665d2737172b12', 'request_id': 'req-51ed14fa-6375-482c-a281-7be6aa19abfc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.385144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9daf941830f4a31b8665d2737172b12', 'request_id': 'req-51ed14fa-6375-482c-a281-7be6aa19abfc'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.395413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ceb0dd56-dd3a-48ed-b0cb-691637fe41d8 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:59.396662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ceb0dd56-dd3a-48ed-b0cb-691637fe41d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.415748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] The token cache is being invalidate because project c9daf941830f4a31b8665d2737172b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:52:59.416171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.417775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-51ed14fa-6375-482c-a281-7be6aa19abfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:52:59.424281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1076/2139] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:52:59 2022] DELETE /identity/v3/projects/c9daf941830f4a31b8665d2737172b12 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:52:59.524241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1064/2140] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:59.533449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.534569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.534977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.575247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:59.576560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.579687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b974d-dc25-431b-9f03-34794bbf1a98 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.580401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b974d-dc25-431b-9f03-34794bbf1a98 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.580932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b974d-dc25-431b-9f03-34794bbf1a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.606809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1077/2141] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:59.609730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.620537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa5188-20d8-4545-978b-1152d6a3380f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.644128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.645260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.645741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.646150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.647097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:52:59.647510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.649472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.665329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:59.666167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:52:59.674322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1078/2142] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:52:59.675981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.686988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-758b974d-dc25-431b-9f03-34794bbf1a98 None None] MFA Rules not processed for user `06f644c1c44d4dafacbd5bbdde323ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:59.688716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-758b974d-dc25-431b-9f03-34794bbf1a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.696651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ce9f5-048f-43c6-9f51-efd3dbb029b1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.702928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.703806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.704205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.705086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.705906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.708104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.759422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1079/2143] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:59.760532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.810174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1065/2144] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:52:59.816700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.817000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.817333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.838279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] MFA Rules not processed for user `06f644c1c44d4dafacbd5bbdde323ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:52:59.839892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.851627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af412468-06c2-4832-92aa-009ab6d08096 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.856631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.857318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.857605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.857888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.858685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.861449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.893311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1080/2145] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:52:59.894565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.906541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8dd68-a2b9-4c78-bfef-aab415e04aa0 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.913416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.914659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.915142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.915759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.916803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.918871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.930251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1066/2146] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:52:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:52:59.932773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1081/2147] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:59.936705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.941346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.943231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:52:59.947554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e420e1-b745-44d5-b73c-4c41dc0b264c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.952550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.954434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.954738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.955188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:52:59.955955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:52:59.957605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:52:59.968836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1067/2148] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:52:59.972672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:52:59.983796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0dca1-f7d6-4cc7-b379-b3d61d161b39 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:52:59.988810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:52:59.989719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:52:59.990067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:52:59.990495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] PATH_INFO: `/v3/projects/fa7c4a2e1e294610b96fe4625db19484/users/933ae8e691c94fa195561fa36fc3d062/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:00.009679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7c4a2e1e294610b96fe4625db19484, user_id=933ae8e691c94fa195561fa36fc3d062, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:00.013052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda766ca-1b8a-4a78-8065-13d53db4af02 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:00.028543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1068/2149] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:52:59 2022] PUT /identity/v3/projects/fa7c4a2e1e294610b96fe4625db19484/users/933ae8e691c94fa195561fa36fc3d062/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:00.040902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:00.041202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:00.041486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:00.052660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f9e7-ddc4-4b32-b4de-4cf1c94155f5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:00.071056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] MFA Rules not processed for user `933ae8e691c94fa195561fa36fc3d062`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:00.072295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:00.108942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:00.128994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:00.130416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:00.130836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:00.131425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:00.153746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1069/2150] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:00.167016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:00.168506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:00.174153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:00.223933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:00.225642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:00.236498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1082/2151] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:52:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:00.241645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3daefa-17f6-446b-98bc-b13d1a3ccbe9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:00.247718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:00.248385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:00.248674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:00.248953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:00.259357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:00.261485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ba14e-31db-4834-aaa2-6c56477a425c service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:00.272786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1070/2152] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:00.814043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:00.816018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:00.928413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa72a5f-f9e9-4094-ba02-3384376cec89 service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:00.933801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:00.934741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:00.935029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:00.935314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] PATH_INFO: `/v3/users/1db5afe984724a74b6dfe0960c9ce858` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:00.936534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db5afe984724a74b6dfe0960c9ce858)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:00.938912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:01.006837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db5afe984724a74b6dfe0960c9ce858'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:01.014896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db5afe984724a74b6dfe0960c9ce858'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:01.019475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db5afe984724a74b6dfe0960c9ce858'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:01.020689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db5afe984724a74b6dfe0960c9ce858'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:01.024031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:01.024526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:01.024620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:01.032886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e536d9a-21b4-4273-aec3-364f977b668a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db5afe984724a74b6dfe0960c9ce858'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:01.056173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1083/2153] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:00 2022] DELETE /identity/v3/users/1db5afe984724a74b6dfe0960c9ce858 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:01.060921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:01.061292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:01.061626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:01.070832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] MFA Rules not processed for user `06f644c1c44d4dafacbd5bbdde323ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:01.072618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:01.081518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:01.083719 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:01.090203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:01.158032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1071/2154] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:01.160095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1084/2155] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:01.170763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:01.172579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:01.278364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ae10a-98e0-4272-a7ab-b403cf6c5b87 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:01.285469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:01.287165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:01.287669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:01.288203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:01.300122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:01.302131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:01.314216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1072/2156] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:02.190697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:02.194451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:02.212773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d23a6-6762-40bc-b9c5-5a5b0e738fcc None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:02.222072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:02.223959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:02.224618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:02.225086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] PATH_INFO: `/v3/projects/df5b29d47d554c588abfc86dfbb83b58` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:02.226865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5b29d47d554c588abfc86dfbb83b58)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:02.229135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:02.412858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5b29d47d554c588abfc86dfbb83b58', 'request_id': 'req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.441623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5b29d47d554c588abfc86dfbb83b58', 'request_id': 'req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.458686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5b29d47d554c588abfc86dfbb83b58', 'request_id': 'req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.505415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] The token cache is being invalidate because project df5b29d47d554c588abfc86dfbb83b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:02.505720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.507981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.511206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1085/2157] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:02 2022] DELETE /identity/v3/projects/df5b29d47d554c588abfc86dfbb83b58 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:02.515575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:02.529171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0176537-4ef6-428e-bccf-6d8fcc4b7471 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:02.609511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:02.621785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:02.624903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:02.625206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:02.625493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] PATH_INFO: `/v3/users/30a3e07242f34edea8e3c1e6793043b7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:02.627061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a3e07242f34edea8e3c1e6793043b7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:02.629068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:02.734730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a3e07242f34edea8e3c1e6793043b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.764204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a3e07242f34edea8e3c1e6793043b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.767027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a3e07242f34edea8e3c1e6793043b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.771262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a3e07242f34edea8e3c1e6793043b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.793417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1fde77f1-34c8-413e-8f40-f2ca046fc9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a3e07242f34edea8e3c1e6793043b7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:02.814723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1073/2158] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:02 2022] DELETE /identity/v3/users/30a3e07242f34edea8e3c1e6793043b7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:03.164442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.169516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:03.190449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c98c-e8ca-4ad6-9102-448f7d0ef3f6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.338383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.343602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.345142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.345616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] PATH_INFO: `/v3/projects/3568da27f7274c1ab661e60402201c33` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.350862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3568da27f7274c1ab661e60402201c33)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.353264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.450827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:03.469461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3568da27f7274c1ab661e60402201c33', 'request_id': 'req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:03.480404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3568da27f7274c1ab661e60402201c33', 'request_id': 'req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:03.487860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3568da27f7274c1ab661e60402201c33', 'request_id': 'req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:03.500634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] The token cache is being invalidate because project 3568da27f7274c1ab661e60402201c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:03.500935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:03.501648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-25258d1c-a475-4e5f-aa58-a6db56ee4c8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:03.508449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.508609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.508930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.510180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1086/2159] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:03 2022] DELETE /identity/v3/projects/3568da27f7274c1ab661e60402201c33 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:03.534979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:03.536944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:03.619341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1074/2160] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:03.628257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.628556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.628845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.649847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:03.651135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:03.660541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1087/2161] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:03.664718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.674926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e763fa-aadc-4abe-a596-c46b37e83422 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.680025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.680843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.681189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.681533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.682384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:03.682791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.684476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.697265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1075/2162] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:53:03.701788 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.712497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c5444-75bd-4e73-869b-0a09aaadf840 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.717366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.718068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.718359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.718638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.719356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.721292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.751988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1088/2163] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:03 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:03.753478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.764351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102317a5-b1ce-4aed-b977-4f14cc665c40 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.824667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.825445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.825734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.826044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.826784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.828756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.836386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:03.852118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1076/2164] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:03 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:03.856320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.867367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12335809-84cd-489a-a57d-f363b97ae3ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.872166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.872837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.873120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.873409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.874181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.875829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.887380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1089/2165] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:03.891793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.903271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c05c5a-612d-493c-abcc-37be26c798c3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.908193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.908858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.909139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.909414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.910183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.911903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aad59-4c99-4b4a-a764-d1230f2e213e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.924414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1077/2166] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:03.928684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:03.939326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f574993-4a77-4e73-b5f2-389bb041a0d8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:03.944405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:03.945301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.945586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.945864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] PATH_INFO: `/v3/projects/e1527c96c2dd40f98ea81796703217ae/users/af362496e1db4c46afdc84aa89c49839/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:03.961417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1527c96c2dd40f98ea81796703217ae, user_id=af362496e1db4c46afdc84aa89c49839, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:03.964149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:03.977830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1090/2167] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:03 2022] PUT /identity/v3/projects/e1527c96c2dd40f98ea81796703217ae/users/af362496e1db4c46afdc84aa89c49839/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:03.988202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:03.988493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:03.988809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.017483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] MFA Rules not processed for user `af362496e1db4c46afdc84aa89c49839`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:04.018741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:04.088340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1078/2168] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:04.093010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.158672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cde8eb-b7c1-49ea-b353-76ac6fe8e352 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.163794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.164550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.164958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.165247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.165976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.167932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.201265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1091/2169] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:04 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:04.202477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.213986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6995e-0a4d-447c-9a2c-c3c76fd5c4b1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.276450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.277125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.277409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.277686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.278439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.280386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.302276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1079/2170] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:04 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:04.306403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.318056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17240e3-6a59-40d1-b9b7-0363cb6d4f77 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.324359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.325092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.325308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.325575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.326337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.328197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.339673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1092/2171] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:04.343915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.354576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10521691-864e-4ed0-a303-74628b3b4cd5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.359267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.360122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.360402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.360674 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] PATH_INFO: `/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c/users/4de7127fd75743648d1756deffcd8cd7/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.375927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f52dbd82bc4227b6cdb99eb4838a6c, user_id=4de7127fd75743648d1756deffcd8cd7, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.378759 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.392404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1080/2172] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:04 2022] PUT /identity/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c/users/4de7127fd75743648d1756deffcd8cd7/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:04.396529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.408899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a8061-1d55-4927-b729-581e900cfe7d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.447075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:04.473563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.474273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.474575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.474840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.475561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.477167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93de208-75d9-43c0-a524-80e82fb18691 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.486729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1093/2173] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:04.491369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.501673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e6c2d-2a83-455c-bb6e-673f3e98f792 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.506961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.507827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.508112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.508388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] PATH_INFO: `/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c/users/4de7127fd75743648d1756deffcd8cd7/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.510394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f52dbd82bc4227b6cdb99eb4838a6c, user_id=4de7127fd75743648d1756deffcd8cd7, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.513048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.523690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1081/2174] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:04 2022] PUT /identity/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c/users/4de7127fd75743648d1756deffcd8cd7/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:04.534220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.534512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.534792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.561088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] MFA Rules not processed for user `4de7127fd75743648d1756deffcd8cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:04.562334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:04.628595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1094/2175] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:04.640173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:04.642124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:04.672008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:04.717197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f59fd-fd28-4fd8-847d-f19654ac326d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:04.720949 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:04.721671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:04.721974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:04.722274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:04.779500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:04.780968 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e92e7f-9616-4a37-a9b6-f6b6b01828aa service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:04.790830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1082/2176] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:06.578887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:06.581261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:06.674930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf5525-8f81-4ffc-9169-f02795079f88 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:06.682071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:06.683255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:06.683602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:06.683942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] PATH_INFO: `/v3/users/f16693cdac034450bc435f645ca4be58` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:06.685234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16693cdac034450bc435f645ca4be58)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:06.687295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:06.743251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16693cdac034450bc435f645ca4be58'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:06.760326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16693cdac034450bc435f645ca4be58'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:06.761501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16693cdac034450bc435f645ca4be58'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:06.762262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16693cdac034450bc435f645ca4be58'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:06.765731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:06.774154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16693cdac034450bc435f645ca4be58'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:06.797991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1095/2177] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:06 2022] DELETE /identity/v3/users/f16693cdac034450bc435f645ca4be58 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:06.803107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:06.803591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:06.803669 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:06.822945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:06.824208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:06.904468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1083/2178] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:06.914641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:06.916042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:06.981180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7abb-d765-4e92-9d03-859b307d4ab4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:06.985131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:06.985735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:06.986103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:06.986381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.001682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.003201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308aaa4-ad94-41c1-b61c-751e249ac317 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.014193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1096/2179] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:07.287021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.288540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:07.300597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2cbec-811f-4142-be87-9f834689e5db None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.305313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.306207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.306494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.306771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] PATH_INFO: `/v3/projects/740a3934b7a2462fa458231439b4a3aa` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.308045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=740a3934b7a2462fa458231439b4a3aa)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.310010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.356923 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740a3934b7a2462fa458231439b4a3aa', 'request_id': 'req-149c6c4f-0b82-4267-8767-12b8f5802441'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:07.363034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740a3934b7a2462fa458231439b4a3aa', 'request_id': 'req-149c6c4f-0b82-4267-8767-12b8f5802441'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:07.368484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740a3934b7a2462fa458231439b4a3aa', 'request_id': 'req-149c6c4f-0b82-4267-8767-12b8f5802441'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:07.376806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:07.382409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] The token cache is being invalidate because project 740a3934b7a2462fa458231439b4a3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:07.382859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:07.383764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-149c6c4f-0b82-4267-8767-12b8f5802441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:07.386151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1084/2180] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:07 2022] DELETE /identity/v3/projects/740a3934b7a2462fa458231439b4a3aa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:07.391132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.391422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.391702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.409265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:07.410478 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:07.474911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1097/2181] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:07.484498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.484798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.485080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.503032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:07.504243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:07.512050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1085/2182] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:07.516172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.526401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14b8aa-a16e-48d9-b9d5-bbe86974f790 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.531130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.531751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.532033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.532304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.533057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:07.533356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.535068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.547070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1098/2183] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:53:07.551956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.562021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d05bc-1f65-47d1-ad42-c04e100d3b43 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.566724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.567394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.567679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.567956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.568648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.570616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.597533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1086/2184] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:07 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:07.604572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.615680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc45c850-1c83-4560-b9fd-daeaf6fafe6b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.675289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.675963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.676251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.676657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.677410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.679369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.690987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:07.709980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1099/2185] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:07 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:07.713787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.724547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c090755-d383-48a6-bf66-1bf6a8840b0f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.729297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.729973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.730268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.730777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.731851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.733476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.744354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1087/2186] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:07.748424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.759210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5320c-a734-4d1a-938f-06c08cba743f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.764021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.764964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.765253 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.765550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.766421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.768474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819c25-6f30-4738-966e-1ea2bd07ddf4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.781301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1100/2187] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:07.785448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.795637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e31b1-e468-43e9-9aed-3f5348ca3645 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:07.800438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:07.801362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.801662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.801933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] PATH_INFO: `/v3/projects/62b4b973f179428684fb300088943945/users/ab6d9902f9fd4043969aa2396481f54e/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.817361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b4b973f179428684fb300088943945, user_id=ab6d9902f9fd4043969aa2396481f54e, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:07.820109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:07.833826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1088/2188] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:07 2022] PUT /identity/v3/projects/62b4b973f179428684fb300088943945/users/ab6d9902f9fd4043969aa2396481f54e/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:07.844671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:07.845203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:07.845486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:07.876255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] MFA Rules not processed for user `ab6d9902f9fd4043969aa2396481f54e`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:07.877470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:07.949422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1101/2189] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:07.962868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:07.964637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:08.025208 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa61a2-2d1e-4037-b74d-63c44ff8a600 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:08.029101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:08.029756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:08.030076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:08.030359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:08.038978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:08.040427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:08.049543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1089/2190] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:11.150259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:11.164583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a61d1-52e2-4a4d-8f2f-4e0b9599c35f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:11.170003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:11.170003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:11.170003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:11.170003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:11.172870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:11.256126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:11.257756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee892f44-3db0-4ad1-b9a4-0dd3965f1b08 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:11.271114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1102/2191] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:18.045248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:21.264926 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=69 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:21.335335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:21.408140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa1915-344a-49a9-b831-b40fb6e9f365 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:21.413583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:21.414532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:21.414854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:21.415106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] PATH_INFO: `/v3/users/ab6d9902f9fd4043969aa2396481f54e` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:21.416898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6d9902f9fd4043969aa2396481f54e)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:21.420373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:21.479626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6d9902f9fd4043969aa2396481f54e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:21.494689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6d9902f9fd4043969aa2396481f54e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:21.495408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6d9902f9fd4043969aa2396481f54e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:21.496010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6d9902f9fd4043969aa2396481f54e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:21.504471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6d9902f9fd4043969aa2396481f54e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:21.517273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1090/2192] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:21 2022] DELETE /identity/v3/users/ab6d9902f9fd4043969aa2396481f54e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:21.522786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:21.523068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:21.523351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:21.542090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:21.543313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:21.623956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1103/2193] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:21.631542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:21.633509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:21.737526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bed089-0ba7-455b-9818-3afc661a277b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:21.741457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:21.742311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:21.742530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:21.742756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:21.754100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:21.755906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:21.775104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1091/2194] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:22.046879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:22.048542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:22.061358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede837-0209-48a0-b6b2-51263b0e9991 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:22.067988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:22.068934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:22.069296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:22.069640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] PATH_INFO: `/v3/projects/62b4b973f179428684fb300088943945` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:22.071098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b4b973f179428684fb300088943945)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:22.073317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:22.121865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b4b973f179428684fb300088943945', 'request_id': 'req-b036e306-0456-4c3a-85bb-1c079ba7f154'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.130009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b4b973f179428684fb300088943945', 'request_id': 'req-b036e306-0456-4c3a-85bb-1c079ba7f154'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.137184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b4b973f179428684fb300088943945', 'request_id': 'req-b036e306-0456-4c3a-85bb-1c079ba7f154'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.148459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] The token cache is being invalidate because project 62b4b973f179428684fb300088943945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:22.148749 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.149394 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b036e306-0456-4c3a-85bb-1c079ba7f154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.150984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1104/2195] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:22 2022] DELETE /identity/v3/projects/62b4b973f179428684fb300088943945 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:22.487211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:22.489479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:22.502153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f40be-8778-4994-aa03-18f7d8b56d93 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:22.521760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:22.564910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:22.567362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:22.567969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:22.568084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] PATH_INFO: `/v3/users/af362496e1db4c46afdc84aa89c49839` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:22.569591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af362496e1db4c46afdc84aa89c49839)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:22.571818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:22.617555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af362496e1db4c46afdc84aa89c49839'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.629808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af362496e1db4c46afdc84aa89c49839'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.630507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af362496e1db4c46afdc84aa89c49839'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.631120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af362496e1db4c46afdc84aa89c49839'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.638207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af362496e1db4c46afdc84aa89c49839'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:22.650642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1092/2196] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:22 2022] DELETE /identity/v3/users/af362496e1db4c46afdc84aa89c49839 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:22.655012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:22.655458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:22.655741 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:22.673839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:22.675074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:22.732898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1105/2197] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:22.743115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:22.744499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:22.803646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6ae85-ef58-4564-839b-aa4d3c7ff555 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:22.807606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:22.808250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:22.808536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:22.808815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:22.820121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:22.821811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:22.832810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1093/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:23.106780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:23.108216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:23.120057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dee00-cad8-4f88-bdad-a5cf8f227776 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:23.124704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:23.125762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.126078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.126351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] PATH_INFO: `/v3/projects/e1527c96c2dd40f98ea81796703217ae` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.127708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1527c96c2dd40f98ea81796703217ae)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:23.129695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:23.151742 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:23.178401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1527c96c2dd40f98ea81796703217ae', 'request_id': 'req-7b8e29b9-3a18-4ede-84f8-470357dc2a62'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.185723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1527c96c2dd40f98ea81796703217ae', 'request_id': 'req-7b8e29b9-3a18-4ede-84f8-470357dc2a62'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.192166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1527c96c2dd40f98ea81796703217ae', 'request_id': 'req-7b8e29b9-3a18-4ede-84f8-470357dc2a62'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.208081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] The token cache is being invalidate because project e1527c96c2dd40f98ea81796703217ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:23.208367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.209013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.210621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1106/2199] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:23 2022] DELETE /identity/v3/projects/e1527c96c2dd40f98ea81796703217ae => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:23.214691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:23.224867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31510965-739e-439b-8da6-1cac3641bec1 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:23.265939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:23.283551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:23.284422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.284704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.285028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] PATH_INFO: `/v3/users/4de7127fd75743648d1756deffcd8cd7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.286838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de7127fd75743648d1756deffcd8cd7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:23.288810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:23.335584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de7127fd75743648d1756deffcd8cd7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.347201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de7127fd75743648d1756deffcd8cd7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.348403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de7127fd75743648d1756deffcd8cd7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.349014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de7127fd75743648d1756deffcd8cd7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.357027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de7127fd75743648d1756deffcd8cd7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.373152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1094/2200] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:23 2022] DELETE /identity/v3/users/4de7127fd75743648d1756deffcd8cd7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:23.725833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:23.727399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:23.743138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e29b9-3a18-4ede-84f8-470357dc2a62 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:23.802585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:23.803470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.803756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.804033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] PATH_INFO: `/v3/users/933ae8e691c94fa195561fa36fc3d062` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.805292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933ae8e691c94fa195561fa36fc3d062)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:23.807324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:23.854771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933ae8e691c94fa195561fa36fc3d062'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.862695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933ae8e691c94fa195561fa36fc3d062'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.863800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933ae8e691c94fa195561fa36fc3d062'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.864411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933ae8e691c94fa195561fa36fc3d062'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.867992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:23.869435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:23.874382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a0a1c2a6-d437-4bd7-be13-9eafc5428936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933ae8e691c94fa195561fa36fc3d062'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.883944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34a4d3-2d05-4dbe-b24b-4aa01d9eddb6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:23.889963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:23.890901 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.891202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.891450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] PATH_INFO: `/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.892692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f52dbd82bc4227b6cdb99eb4838a6c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:23.894783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:23.895287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1107/2201] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:23 2022] DELETE /identity/v3/users/933ae8e691c94fa195561fa36fc3d062 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:23.899496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.899787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.900065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.917057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:23.918325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:23.940999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:23.947165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f52dbd82bc4227b6cdb99eb4838a6c', 'request_id': 'req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.956710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f52dbd82bc4227b6cdb99eb4838a6c', 'request_id': 'req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.963691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f52dbd82bc4227b6cdb99eb4838a6c', 'request_id': 'req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.977977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] The token cache is being invalidate because project 92f52dbd82bc4227b6cdb99eb4838a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:23.978273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.979073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c6dd1bb9-3ace-45f8-9637-e87a82ef548e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:23.984929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1095/2202] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:23 2022] DELETE /identity/v3/projects/92f52dbd82bc4227b6cdb99eb4838a6c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:23.986215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1108/2203] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:23.986579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da0cf-450b-48d9-b030-0b7832d89f25 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:23.986933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da0cf-450b-48d9-b030-0b7832d89f25 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:23.987273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da0cf-450b-48d9-b030-0b7832d89f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:23.996447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:23.997809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.011398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-680da0cf-450b-48d9-b030-0b7832d89f25 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.012620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-680da0cf-450b-48d9-b030-0b7832d89f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.060792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:24.073103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e0489-512f-4694-9985-477bdf7a6ec5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.090856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1096/2204] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:24.092417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.092709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.092985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.116336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.117722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.131808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1097/2205] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:24.140091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.143562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.144216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.144531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.144811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.149140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffda29-504f-4da1-9b6f-a9530719289f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.154121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.154283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.154768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.155054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.155332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.155729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.156057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:24.156370 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.158029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.166016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1109/2206] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:24.174782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1098/2207] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:53:24.176468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.185841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca86aed-5efd-4b35-99f2-89cfc846a2f3 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.190536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.191183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.191498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.191776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.192500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.194505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.221301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1110/2208] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:24.226594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.237704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6967c29-ca8c-4a28-9345-a2ffa16666a9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.299607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.300274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.301302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.301583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.302322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.304258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.329993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1099/2209] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:24.332235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.343347 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6af748-83d9-4223-aed1-0985c4ce67e7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.348639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.349346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.349622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.349908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.350676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.352331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.364809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1111/2210] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:24.368828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.379142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e84668-6f8c-4331-bb8b-d2b74fe024a7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.384507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.385198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.385484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.385758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.386515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.388124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.399050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1100/2211] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:24.403346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.413894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e7a16-4075-44f5-9ada-b6303b6b465b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.418693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.419576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.419866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.420193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] PATH_INFO: `/v3/projects/920528ad6b6e4f2eb12acc81261e4c34/users/de9780897dfc47f883622829f84a5ed3/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.438142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=920528ad6b6e4f2eb12acc81261e4c34, user_id=de9780897dfc47f883622829f84a5ed3, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.441578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b8c01-84d5-4f39-a9dd-93c14c7f92aa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.443053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.445793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.457830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1112/2212] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:24 2022] PUT /identity/v3/projects/920528ad6b6e4f2eb12acc81261e4c34/users/de9780897dfc47f883622829f84a5ed3/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:24.462351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178c31d-ed59-4101-a568-a786bc6065c5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.469214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484787-633d-40e3-b51c-17064f909e4d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.469623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484787-633d-40e3-b51c-17064f909e4d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.469785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484787-633d-40e3-b51c-17064f909e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.497963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-00484787-633d-40e3-b51c-17064f909e4d None None] MFA Rules not processed for user `de9780897dfc47f883622829f84a5ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.499250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-00484787-633d-40e3-b51c-17064f909e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.516228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:24.526478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.527682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.527682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.527902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] PATH_INFO: `/v3/projects/fa7c4a2e1e294610b96fe4625db19484` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.529074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7c4a2e1e294610b96fe4625db19484)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.531046 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.576238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1113/2213] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:24.583951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:24.585119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7c4a2e1e294610b96fe4625db19484', 'request_id': 'req-afabfa4b-d0a1-4fe3-8aee-eaa245307513'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.588775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484787-633d-40e3-b51c-17064f909e4d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.590450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-00484787-633d-40e3-b51c-17064f909e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.598738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7c4a2e1e294610b96fe4625db19484', 'request_id': 'req-afabfa4b-d0a1-4fe3-8aee-eaa245307513'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.604752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7c4a2e1e294610b96fe4625db19484', 'request_id': 'req-afabfa4b-d0a1-4fe3-8aee-eaa245307513'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.618894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] The token cache is being invalidate because project fa7c4a2e1e294610b96fe4625db19484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:24.619286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.620016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-afabfa4b-d0a1-4fe3-8aee-eaa245307513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.622173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1101/2214] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:24 2022] DELETE /identity/v3/projects/fa7c4a2e1e294610b96fe4625db19484 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:24.627599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f8fc-ae63-4d99-a4d4-caf23d7f18ca None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.628165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f8fc-ae63-4d99-a4d4-caf23d7f18ca None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.628451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3f8fc-ae63-4d99-a4d4-caf23d7f18ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.649264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b6c3f8fc-ae63-4d99-a4d4-caf23d7f18ca None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.650521 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b6c3f8fc-ae63-4d99-a4d4-caf23d7f18ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.653215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484787-633d-40e3-b51c-17064f909e4d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.705870 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.706671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.707504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.707784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.709688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.720308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1102/2215] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:24.728335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.728632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.729030 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.747147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.748375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.757938 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1103/2216] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:24.762157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.772726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b05414-d126-4b32-97a2-97320ad9dc7a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.776626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.778149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.778676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.779488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.779926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.780357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.781225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:24.781795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.783702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.788293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1114/2217] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:24.801997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1104/2218] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:53:24.806776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.807027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.809864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.817325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2586-5d7e-4304-9a8e-e1774682a7b7 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.822762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.823437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.823701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.824194 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.824940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.826065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731a95-ed1d-4954-82ca-91edd439d865 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.827184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.831419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.833658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.834040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.834296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] PATH_INFO: `/v3/users/de9780897dfc47f883622829f84a5ed3` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.835706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9780897dfc47f883622829f84a5ed3)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.838173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.857441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1115/2219] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:24.861843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.871786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:24.915893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9780897dfc47f883622829f84a5ed3'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.929872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9780897dfc47f883622829f84a5ed3'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.930634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9780897dfc47f883622829f84a5ed3'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.931274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9780897dfc47f883622829f84a5ed3'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.939180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bd7b0-2524-4ff7-bd3c-a75f4d8222ee admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.943306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0a1038f4-4b5e-4e51-949e-837a1a8e5091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9780897dfc47f883622829f84a5ed3'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:24.945035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:24.945853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.946167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.946451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.947165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:24.949364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:24.960808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1105/2220] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:24 2022] DELETE /identity/v3/users/de9780897dfc47f883622829f84a5ed3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:24.965435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:24.966050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:24.966392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:24.973026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1116/2221] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:24.976888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:24.978773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:24.991034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:24.992069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83ec6a-1d6d-4e04-ab3e-6bf675b2330c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:24.992440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.029805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:25.077728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.078400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.078688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.078995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.079756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.081476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.086670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1106/2222] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:25.096260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1117/2223] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:53:25.096676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.097600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.097997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.108851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0e4ab-224e-42d0-b0ac-0ff1306a47d5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.114960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.115757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.116134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.116488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.117302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.119256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.134246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1118/2224] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:25.136776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.152020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de2747-d198-465b-a2aa-aea55895e722 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.158048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.159081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.159369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.159700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] PATH_INFO: `/v3/projects/b3dc696d309b4e2fb584bde9330dd8bd/users/cf0e5fec45f24e9fb1dad97e948dbd73/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.178803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3dc696d309b4e2fb584bde9330dd8bd, user_id=cf0e5fec45f24e9fb1dad97e948dbd73, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.181736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341ecba-619c-46e6-b61a-bd51498aefb9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.199581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1119/2225] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:25 2022] PUT /identity/v3/projects/b3dc696d309b4e2fb584bde9330dd8bd/users/cf0e5fec45f24e9fb1dad97e948dbd73/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:25.211866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.212041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47768-9c83-4cd7-a7de-924b2fbc79ae None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.212287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.212673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.258453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] MFA Rules not processed for user `cf0e5fec45f24e9fb1dad97e948dbd73`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:25.259754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.272883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.273661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.274042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.274351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.316052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:25.341455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.342952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.349122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1120/2226] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:25.356353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1107/2227] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:25.357925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.359482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.432861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8d2f-63ff-46a9-873f-1834b586e7e0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.437296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.437982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.438268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.438548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.448776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.450405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f8461-0c40-4935-9456-bf0e1fdf62d5 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.461116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1121/2228] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:25.488109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.499788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba21e3f-4476-4dd3-93b8-cb5fcfc0cbc7 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.505314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.506535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.506899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.507254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] PATH_INFO: `/v3/users/cf0e5fec45f24e9fb1dad97e948dbd73` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.508568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0e5fec45f24e9fb1dad97e948dbd73)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.511224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.555460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:25.574363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0e5fec45f24e9fb1dad97e948dbd73'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.587867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0e5fec45f24e9fb1dad97e948dbd73'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.588536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0e5fec45f24e9fb1dad97e948dbd73'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.589173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0e5fec45f24e9fb1dad97e948dbd73'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.597018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0e5fec45f24e9fb1dad97e948dbd73'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.610025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1108/2229] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:25 2022] DELETE /identity/v3/users/cf0e5fec45f24e9fb1dad97e948dbd73 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:25.614531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.614820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.615097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.632854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:25.634163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.698092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1122/2230] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:25.708883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.710333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.774231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec171c-7b5c-4eaa-b180-6e5db0c5fe08 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.779638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:25.781131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:25.781368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.782054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.782400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.782681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.798818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.800304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0c1ca-7b2d-4532-aae2-6cd3644b0a17 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.801313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5587b-5ae0-48e7-ac41-d08ec2e25328 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:25.807953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:25.809048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.809407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.809872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] PATH_INFO: `/v3/projects/920528ad6b6e4f2eb12acc81261e4c34` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.812069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1109/2231] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:53:25.821046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=920528ad6b6e4f2eb12acc81261e4c34)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:25.827255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:25.897976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '920528ad6b6e4f2eb12acc81261e4c34', 'request_id': 'req-15c6d497-db39-4543-985c-0e0bed1f2f3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.910569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '920528ad6b6e4f2eb12acc81261e4c34', 'request_id': 'req-15c6d497-db39-4543-985c-0e0bed1f2f3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.920461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '920528ad6b6e4f2eb12acc81261e4c34', 'request_id': 'req-15c6d497-db39-4543-985c-0e0bed1f2f3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.936068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] The token cache is being invalidate because project 920528ad6b6e4f2eb12acc81261e4c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:25.936363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.937318 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-15c6d497-db39-4543-985c-0e0bed1f2f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:25.942010 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1123/2232] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:25 2022] DELETE /identity/v3/projects/920528ad6b6e4f2eb12acc81261e4c34 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:25.945239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:25.945704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:25.945792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:25.965561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:25.967282 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:26.034356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1110/2233] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:26.043008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.043295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.043570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.065267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:26.067171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:26.075089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1124/2234] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:26.079699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.091312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75175190-a2ae-4908-9f6b-07cc8e3f216c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.096916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.097670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.097982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.098273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.099105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:26.099404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.101099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.118185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1111/2235] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:53:26.119598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.121277 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:26.126783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.134883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce7a16-93de-4f9e-8c09-a25ce60e2c7b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.139802 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.140779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.141120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.141403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] PATH_INFO: `/v3/projects/b3dc696d309b4e2fb584bde9330dd8bd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.142641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3dc696d309b4e2fb584bde9330dd8bd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.144757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.147087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:26.150299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8accc4-c54e-4764-ab1a-d6be0702b15a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.158755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.159523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.159910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.160264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.161041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.163132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.193763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1112/2236] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:26.197872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.210296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3dc696d309b4e2fb584bde9330dd8bd', 'request_id': 'req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:26.215314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:26.220431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3dc696d309b4e2fb584bde9330dd8bd', 'request_id': 'req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:26.228643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3dc696d309b4e2fb584bde9330dd8bd', 'request_id': 'req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:26.254563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] The token cache is being invalidate because project b3dc696d309b4e2fb584bde9330dd8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:26.254859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:26.255546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:26.257397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1125/2237] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:26 2022] DELETE /identity/v3/projects/b3dc696d309b4e2fb584bde9330dd8bd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:26.296207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da5c95-8d07-4823-92a9-25da9b01af3e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.318005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.318742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.319029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.319310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.320023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.322150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.345587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1113/2238] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:26.349899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.351439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:26.422787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ade54-13b9-4845-a1a8-9bf8ed1d2c46 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.428487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.429207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.429498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.429779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.430558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.432214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.446276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1126/2239] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:26.447579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.457728 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2152ba-e361-4158-bc1a-946098d8bdfd admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.462432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.463087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.463376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.463657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.464383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.466061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea148be7-f651-48c1-809e-358c04589642 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.476785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1114/2240] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:26.480518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.490932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6c1f6-52ca-4372-8d43-0e5af2d99d94 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.495720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.496599 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.496887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.497167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] PATH_INFO: `/v3/projects/8ea63dad750647018540b329aa16bd82/users/aca9884d42a44d38a5760ca72ee67700/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.511809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea63dad750647018540b329aa16bd82, user_id=aca9884d42a44d38a5760ca72ee67700, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.514552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.527423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1127/2241] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:26 2022] PUT /identity/v3/projects/8ea63dad750647018540b329aa16bd82/users/aca9884d42a44d38a5760ca72ee67700/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:53:26.537832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.538151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.538430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.564429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] MFA Rules not processed for user `aca9884d42a44d38a5760ca72ee67700`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:26.565620 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:26.631050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1115/2242] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:26.635760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.703125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3702a18-0ce7-4db7-ae9c-5b809fb0203b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.707985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.708206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.708493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.708770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.709472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.711440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.715962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:26.741555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1128/2243] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:26.745504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.756678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465d2fa-2c9b-4d35-800b-4977fc3b12f9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.812120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.812780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.813063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.813339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.814055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.816025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.836051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1116/2244] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:26.839862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.851007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b75bb5-d8d3-4b21-98e5-e04133656612 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.855563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.856210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.856503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.856779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.857505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.859137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.869443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1129/2245] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:53:26.873171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.883683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620656a-5c3a-4da3-b8f9-522999063b95 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.888401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.889064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.889347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.889619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.890379 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.891971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fe4cc-db51-45d0-b6c7-b807a93b727a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.902594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1117/2246] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:26.906498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:26.916943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea50425-1ff6-4293-8dce-08d804d18ca8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:26.921634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:26.922534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.922822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.923096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] PATH_INFO: `/v3/projects/eacebc21b1fd4fb08eb699be020f3c2d/users/67cc0ac471074d0a85264b914ffd8ee4/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.937826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacebc21b1fd4fb08eb699be020f3c2d, user_id=67cc0ac471074d0a85264b914ffd8ee4, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:26.940489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:26.953167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1130/2247] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:26 2022] PUT /identity/v3/projects/eacebc21b1fd4fb08eb699be020f3c2d/users/67cc0ac471074d0a85264b914ffd8ee4/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:26.963610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:26.964103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:26.964526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:26.970293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:26.993127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] MFA Rules not processed for user `67cc0ac471074d0a85264b914ffd8ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:26.994395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:27.064482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1118/2248] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:27.069261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.129472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fdf58-3ba3-4c70-b9e1-053573dec77e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.134086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.134730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.135012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.135288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.135979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.137883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.162550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1131/2249] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:27 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:27.166577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.178037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373191a7-67ff-4e76-a47c-403dc3300f57 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.234033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.234634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.234920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.235202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.235910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.237874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.259456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1119/2250] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:27 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:27.263437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.274678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1e18-c638-48d2-a48a-984ddc455b37 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.279376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.280034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.280339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.280657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.281356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.283007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.294040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1132/2251] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:27.298169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.308211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6205b-e8e1-4bf6-843f-2e9442eab35c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.312875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.313750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.314072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.314356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] PATH_INFO: `/v3/projects/db73c03a7bf04a73ac461da7b8ade39f/users/bfd231dcdebf403bb17738bd299283b4/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.336127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db73c03a7bf04a73ac461da7b8ade39f, user_id=bfd231dcdebf403bb17738bd299283b4, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.340636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.363806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1120/2252] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:27 2022] PUT /identity/v3/projects/db73c03a7bf04a73ac461da7b8ade39f/users/bfd231dcdebf403bb17738bd299283b4/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:27.367969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.382434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813053e-c681-49d5-9d78-f6362aec03d1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.445752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.446590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.446874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.447153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.447884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.449535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b86978-8d91-428f-a1ed-0572b6668af2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.460407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1133/2253] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:27.464533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.475185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349fd6f6-a745-4376-91bc-d4c411969aa7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.481708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.482993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.483275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.483551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] PATH_INFO: `/v3/projects/db73c03a7bf04a73ac461da7b8ade39f/users/bfd231dcdebf403bb17738bd299283b4/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.485594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db73c03a7bf04a73ac461da7b8ade39f, user_id=bfd231dcdebf403bb17738bd299283b4, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.488832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.499053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1121/2254] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:27 2022] PUT /identity/v3/projects/db73c03a7bf04a73ac461da7b8ade39f/users/bfd231dcdebf403bb17738bd299283b4/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:27.510605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.510906 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.511193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.537649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] MFA Rules not processed for user `bfd231dcdebf403bb17738bd299283b4`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:27.538881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:27.547846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:27.607344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1134/2255] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:27.619479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:27.621167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:27.667015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:27.679432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161665-0880-4908-a575-1271831f0954 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:27.683177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:27.683806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:27.684089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:27.684359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:27.743242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:27.744819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:27.754804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1122/2256] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:29.260454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:29.261896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:29.302623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c89183-dd8a-496c-9d2d-c542ffad425e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:29.306255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:29.306898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:29.307183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:29.307461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:29.376374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:29.377849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8b098-f405-42ad-a942-ef91ef295fe4 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:29.387393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1135/2257] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:29.945860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:29.965478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09f9db-f5fc-4abe-9f31-d00c9d7ab62c service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:29.969272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:29.969911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:29.970230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:29.970506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:29.986996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:29.988550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:29.998190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1123/2258] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:32.817917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:32.818411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:32.818607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:32.837227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:32.838453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:32.892234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1136/2259] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:53:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:32.933516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:32.934960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:33.002388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfe3d7-0b96-49d3-af1f-a5a6cf669e69 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:33.007340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:33.007985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:33.008268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:33.008553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:33.016677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:33.018314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6de79-009a-4206-8987-37516d9db34a service swift] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:33.026757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1124/2260] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:53:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:53:33.917370 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:33.929244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:34.012447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da668b8-70c6-4f21-961a-b1e7d738af56 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:34.017408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:34.018323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:34.018610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:34.018890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] PATH_INFO: `/v3/users/aca9884d42a44d38a5760ca72ee67700` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:34.020687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca9884d42a44d38a5760ca72ee67700)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:34.022688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:34.087238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca9884d42a44d38a5760ca72ee67700'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.095849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca9884d42a44d38a5760ca72ee67700'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.096566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca9884d42a44d38a5760ca72ee67700'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.097175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca9884d42a44d38a5760ca72ee67700'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.110051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca9884d42a44d38a5760ca72ee67700'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.132331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1137/2261] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:33 2022] DELETE /identity/v3/users/aca9884d42a44d38a5760ca72ee67700 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:34.137298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:34.137865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:34.137865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:34.183860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:34.185243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:34.220355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:34.257283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1125/2262] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:34.268534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:34.270772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:34.370024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805bb74d-9e96-4b7d-9287-d25b51a763db admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:34.373887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:34.374574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:34.374860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:34.375138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:34.383193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:34.389606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:34.391767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:34.423871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1138/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:34.821047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:34.822564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:34.855714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7131f6-97c4-4fc6-b2bd-3ad295fa3383 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:34.867909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:34.868919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:34.869266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:34.869611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] PATH_INFO: `/v3/projects/8ea63dad750647018540b329aa16bd82` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:34.871189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea63dad750647018540b329aa16bd82)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:34.873522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:34.951368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea63dad750647018540b329aa16bd82', 'request_id': 'req-2887e935-9570-4904-95a8-8cf2af89b85c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.958860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea63dad750647018540b329aa16bd82', 'request_id': 'req-2887e935-9570-4904-95a8-8cf2af89b85c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.964816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea63dad750647018540b329aa16bd82', 'request_id': 'req-2887e935-9570-4904-95a8-8cf2af89b85c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.985036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] The token cache is being invalidate because project 8ea63dad750647018540b329aa16bd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:34.985341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.986310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:34.993995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1126/2264] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:34 2022] DELETE /identity/v3/projects/8ea63dad750647018540b329aa16bd82 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:34.995243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:35.006377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee54a7-8382-4a5c-b14c-9bc2ca012484 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:35.098263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:35.102767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:35.103549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:35.103839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] PATH_INFO: `/v3/users/67cc0ac471074d0a85264b914ffd8ee4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:35.105278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67cc0ac471074d0a85264b914ffd8ee4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:35.107506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:35.229617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67cc0ac471074d0a85264b914ffd8ee4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.244335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67cc0ac471074d0a85264b914ffd8ee4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.244335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67cc0ac471074d0a85264b914ffd8ee4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.244335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67cc0ac471074d0a85264b914ffd8ee4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.267819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67cc0ac471074d0a85264b914ffd8ee4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.293296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1139/2265] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:34 2022] DELETE /identity/v3/users/67cc0ac471074d0a85264b914ffd8ee4 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:35.637551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:35.638797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:35.658754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887e935-9570-4904-95a8-8cf2af89b85c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:35.740215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:35.741493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:35.741493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:35.741690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] PATH_INFO: `/v3/projects/eacebc21b1fd4fb08eb699be020f3c2d` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:35.743020 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacebc21b1fd4fb08eb699be020f3c2d)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:35.745710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:35.817531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacebc21b1fd4fb08eb699be020f3c2d', 'request_id': 'req-4d0b0d91-008e-4d14-8afd-2f72120204e4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.828672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacebc21b1fd4fb08eb699be020f3c2d', 'request_id': 'req-4d0b0d91-008e-4d14-8afd-2f72120204e4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.839939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacebc21b1fd4fb08eb699be020f3c2d', 'request_id': 'req-4d0b0d91-008e-4d14-8afd-2f72120204e4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.844741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:35.860178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] The token cache is being invalidate because project eacebc21b1fd4fb08eb699be020f3c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:35.860427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.861705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:35.863921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1127/2266] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:35 2022] DELETE /identity/v3/projects/eacebc21b1fd4fb08eb699be020f3c2d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:35.871260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:35.872727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:35.890154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687621-05b6-4bd1-9827-35296a037944 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:35.992453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:35.993913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:35.994592 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:35.995069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] PATH_INFO: `/v3/users/bfd231dcdebf403bb17738bd299283b4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:35.996622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd231dcdebf403bb17738bd299283b4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:35.998680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:36.020491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:36.087629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd231dcdebf403bb17738bd299283b4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.095861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd231dcdebf403bb17738bd299283b4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.097004 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd231dcdebf403bb17738bd299283b4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.098324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd231dcdebf403bb17738bd299283b4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.108439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-658ad1c5-838d-4236-ae14-c322d5e42be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd231dcdebf403bb17738bd299283b4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.133614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1140/2267] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:53:35 2022] DELETE /identity/v3/users/bfd231dcdebf403bb17738bd299283b4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:36.569339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:36.570953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:36.593363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b0d91-008e-4d14-8afd-2f72120204e4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:36.666760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:36.667629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:36.667919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:36.668197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] PATH_INFO: `/v3/projects/db73c03a7bf04a73ac461da7b8ade39f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:36.669453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db73c03a7bf04a73ac461da7b8ade39f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:36.671417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:36.733917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db73c03a7bf04a73ac461da7b8ade39f', 'request_id': 'req-8a1fa610-69f7-4405-9f39-f6a21152e19a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.741726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db73c03a7bf04a73ac461da7b8ade39f', 'request_id': 'req-8a1fa610-69f7-4405-9f39-f6a21152e19a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.751173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db73c03a7bf04a73ac461da7b8ade39f', 'request_id': 'req-8a1fa610-69f7-4405-9f39-f6a21152e19a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.766915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] The token cache is being invalidate because project db73c03a7bf04a73ac461da7b8ade39f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:53:36.767335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.768160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8a1fa610-69f7-4405-9f39-f6a21152e19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:53:36.770455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1128/2268] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:53:36 2022] DELETE /identity/v3/projects/db73c03a7bf04a73ac461da7b8ade39f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:36.775074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:36.777143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:36.777143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:36.828549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:36.829935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:36.901030 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1141/2269] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:36.908557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:36.908893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:36.909268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:36.930874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:36.932616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:36.942332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1129/2270] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:36.946567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:36.958042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9a90-eebe-4540-a277-c7f09345b306 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:36.963482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:36.964270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:36.964665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:36.965042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:36.966064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:53:36.966486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:36.968393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:36.990415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:36.996820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1142/2271] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:53:37.001933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2f5c2-3013-4048-9669-215b92ac8789 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.008051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.008889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.009241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.009657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.010526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.012856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.054457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1130/2272] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:53:36 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:53:37.059797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:37.075387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53809259-92ba-474b-b6a3-9446b6f29a43 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.199644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.199644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.199644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.200242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.201023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.203503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.215602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:37.231849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1143/2273] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:53:37 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:53:37.240541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:37.256976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924f54e-a4bb-49d6-b46b-bb4dff28f4df admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.262389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.263149 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.263498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.264240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.265042 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.269039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.297837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1131/2274] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:37.299232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:37.313272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c364e0-288f-400e-82c0-2c5b08cc7e09 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.319057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.330843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.331303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.331697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.333120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.334876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab20d0-0a15-490d-849e-1293ceeb8d8c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.350814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1144/2275] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:53:37.354472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:37.365151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9f98f-0e38-4f5a-a01e-3e0df453ca3c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.369903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.370812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.371100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.374028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] PATH_INFO: `/v3/projects/b67f4759ef6a47b888079e98fee54fa3/users/e35d08a3e6fe47d7a29a991a44498eaa/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.393939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67f4759ef6a47b888079e98fee54fa3, user_id=e35d08a3e6fe47d7a29a991a44498eaa, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.397844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.417564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1132/2276] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:53:37 2022] PUT /identity/v3/projects/b67f4759ef6a47b888079e98fee54fa3/users/e35d08a3e6fe47d7a29a991a44498eaa/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:53:37.429290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9ef1-b835-4769-b4da-e1680c619d57 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.429657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9ef1-b835-4769-b4da-e1680c619d57 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.430080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f9ef1-b835-4769-b4da-e1680c619d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.464062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ce0f9ef1-b835-4769-b4da-e1680c619d57 None None] MFA Rules not processed for user `e35d08a3e6fe47d7a29a991a44498eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:37.464062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ce0f9ef1-b835-4769-b4da-e1680c619d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:37.567255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1145/2277] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:37.577969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:37.579734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:37.608746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:37.672724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981c97-92a7-4125-8cf0-cfa905817393 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:37.678288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:37.678950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:37.679233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:37.679509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:37.688729 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:37.690210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f9e8-1e5c-45b1-a5d6-80e5bd9eaf80 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:37.705710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1133/2278] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:39.108551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2ed5-f92a-4f2b-a90a-4db328eaae29 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:39.109171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2ed5-f92a-4f2b-a90a-4db328eaae29 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:39.109171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab2ed5-f92a-4f2b-a90a-4db328eaae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:39.133133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e1ab2ed5-f92a-4f2b-a90a-4db328eaae29 None None] MFA Rules not processed for user `e35d08a3e6fe47d7a29a991a44498eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:39.134581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e1ab2ed5-f92a-4f2b-a90a-4db328eaae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:39.143223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1146/2279] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:41.710743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0babdb-e671-418a-8eee-2f2912be568d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:41.711748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0babdb-e671-418a-8eee-2f2912be568d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:41.711954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0babdb-e671-418a-8eee-2f2912be568d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:41.730761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9f0babdb-e671-418a-8eee-2f2912be568d None None] MFA Rules not processed for user `8e98c8ceac6b47079423b45685c5ba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:41.732322 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9f0babdb-e671-418a-8eee-2f2912be568d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:41.800511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1134/2280] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:53:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:49.123587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:INSERT=1,keystone:SELECT=42 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:51.251544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.252120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.252120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.253513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1147/2281] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:53:51.259891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3cc6-b037-4e17-a757-00a8dd802267 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.260377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3cc6-b037-4e17-a757-00a8dd802267 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.260732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c3cc6-b037-4e17-a757-00a8dd802267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.299490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3c1c3cc6-b037-4e17-a757-00a8dd802267 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:51.300795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3c1c3cc6-b037-4e17-a757-00a8dd802267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:51.381710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1135/2282] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:51.393287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:51.395820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:51.451526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124309b-5e46-42c6-90ba-4d5cace60c1a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:51.454927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:51.455565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.455845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.456118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.466504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:51.467942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3904cb1-7522-4499-a335-c93ac5f9c7e5 service glance] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:51.478343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1148/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:51.574289 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.574589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.574874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.576546 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1136/2284] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:53:51.582323 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a796-0d3f-48b1-83d1-ec0cc75f7e0e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.582728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a796-0d3f-48b1-83d1-ec0cc75f7e0e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.583086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9a796-0d3f-48b1-83d1-ec0cc75f7e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.602724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7db9a796-0d3f-48b1-83d1-ec0cc75f7e0e None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:51.604047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7db9a796-0d3f-48b1-83d1-ec0cc75f7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:51.664567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1149/2285] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:53:51.676945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:51.678655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:51.692586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0a483-da34-4b9a-9173-b7995ebb1c60 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:51.696142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:51.696778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.697061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.697340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.704591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:51.705977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77a1f1-0715-45dc-9066-8dbc0dc08df2 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:51.713732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1137/2286] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:51.794448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f477f1-f6d0-4ea2-ab0c-9b0490762b36 None None] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.794738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f477f1-f6d0-4ea2-ab0c-9b0490762b36 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.795013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f477f1-f6d0-4ea2-ab0c-9b0490762b36 None None] PATH_INFO: `/` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.796382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1150/2287] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 10:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 10:53:51.802413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f1366-8f93-412a-90ce-14cf75b15158 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.802709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f1366-8f93-412a-90ce-14cf75b15158 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.802986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f1366-8f93-412a-90ce-14cf75b15158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.843043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:51.851041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a94f1366-8f93-412a-90ce-14cf75b15158 None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:51.852238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a94f1366-8f93-412a-90ce-14cf75b15158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:51.877382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1138/2288] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 10:53:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:51.887626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f477f1-f6d0-4ea2-ab0c-9b0490762b36 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:51.898688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f477f1-f6d0-4ea2-ab0c-9b0490762b36 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:51.902183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:51.902811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:51.903093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:51.903366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:51.913638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:51.915091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:51.925509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1151/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:53:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:53:58.628122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82239-a21a-469c-b7b9-c87ac8a7c7e2 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:58.628589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82239-a21a-469c-b7b9-c87ac8a7c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:58.628679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc82239-a21a-469c-b7b9-c87ac8a7c7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:58.647784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5bc82239-a21a-469c-b7b9-c87ac8a7c7e2 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:58.648974 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5bc82239-a21a-469c-b7b9-c87ac8a7c7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:58.706155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1139/2290] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:53:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:58.718007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:58.719671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:58.790012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b2f9-6f2d-4811-bdaf-59377671b7f1 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:58.795113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:58.795763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:58.796049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:58.796329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:58.805081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:58.806733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:58.814876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1152/2291] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:53:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:53:59.380983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ce6b-3742-4d00-806e-b26b3843e27b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.381275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ce6b-3742-4d00-806e-b26b3843e27b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.381552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ce6b-3742-4d00-806e-b26b3843e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.412549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6855ce6b-3742-4d00-806e-b26b3843e27b None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:59.413986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6855ce6b-3742-4d00-806e-b26b3843e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:59.425164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1140/2292] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:59.440176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:59.451414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146472bc-0e10-444c-aa63-a5fdcc4aaa82 service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:59.456470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:59.457110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.458026 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.458309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.466582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:59.468510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:59.477256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1153/2293] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:53:59.594762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79733d3-0ade-4805-8ffd-92b228813211 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.595112 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79733d3-0ade-4805-8ffd-92b228813211 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.595444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79733d3-0ade-4805-8ffd-92b228813211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.619725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a79733d3-0ade-4805-8ffd-92b228813211 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:59.620944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a79733d3-0ade-4805-8ffd-92b228813211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:59.630499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1141/2294] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:59.644295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:59.662300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1781fc-9661-4474-8db7-4c5e29b821b5 service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:59.667877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:59.668612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.668967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.669313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.679276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:59.680993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:59.689839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1154/2295] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:53:59.734646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b64e-897e-4e33-aafa-e0f5f7834f98 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.735220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b64e-897e-4e33-aafa-e0f5f7834f98 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.735822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b64e-897e-4e33-aafa-e0f5f7834f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.760920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-d6e5b64e-897e-4e33-aafa-e0f5f7834f98 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:53:59.762627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d6e5b64e-897e-4e33-aafa-e0f5f7834f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:53:59.772446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1142/2296] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:53:59.814361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:53:59.821472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:53:59.839632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f0ed5-6896-4991-8ba5-e946dba23392 service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:53:59.845400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:53:59.846587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:53:59.847074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:53:59.847538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:53:59.857236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:53:59.859181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:53:59.869668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1155/2297] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:54:00.391267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:00.391627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:00.392103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:00.413786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:00.415378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:00.425009 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1143/2298] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 10:54:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:00.439256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:00.455205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c338f-362e-44b3-99cb-b9fe970b59cf service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:00.463962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:00.465134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:00.465610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:00.466116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:00.476736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:00.478751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d39220-65e9-48e7-a084-11849545e6dc service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:00.490533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1156/2299] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:54:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:54:02.968714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:03.034803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ab939-c4cb-4538-b8c7-94ee16088ffb None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:03.039807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:03.040678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.040993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.041262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] PATH_INFO: `/v3/users/e35d08a3e6fe47d7a29a991a44498eaa` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.042514 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35d08a3e6fe47d7a29a991a44498eaa)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:03.044476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:03.094666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35d08a3e6fe47d7a29a991a44498eaa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.108200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:03.110539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35d08a3e6fe47d7a29a991a44498eaa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.111676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35d08a3e6fe47d7a29a991a44498eaa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.112747 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35d08a3e6fe47d7a29a991a44498eaa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.121422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35d08a3e6fe47d7a29a991a44498eaa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.134866 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1144/2300] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:54:02 2022] DELETE /identity/v3/users/e35d08a3e6fe47d7a29a991a44498eaa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:54:03.139727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.140290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.140569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.161565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:03.162963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:03.224519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1157/2301] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:03.234124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:03.235684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:03.301516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5ee57-8d44-4c0e-b4ca-7bd31fcdbc8c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:03.314865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:03.315692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.315987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.316268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.331467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:03.333211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a5382-696f-46bc-ae7c-a3b677614846 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:03.348945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1145/2302] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:54:03.607037 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:03.608471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:03.620612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf603b6-896b-4d76-83c4-5324545344c6 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:03.628140 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:03.629476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.629993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.630381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] PATH_INFO: `/v3/projects/b67f4759ef6a47b888079e98fee54fa3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.631859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67f4759ef6a47b888079e98fee54fa3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:03.634601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:03.688955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67f4759ef6a47b888079e98fee54fa3', 'request_id': 'req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.698131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67f4759ef6a47b888079e98fee54fa3', 'request_id': 'req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.705088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67f4759ef6a47b888079e98fee54fa3', 'request_id': 'req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.716667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] The token cache is being invalidate because project b67f4759ef6a47b888079e98fee54fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:54:03.717012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.717804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ddccc665-4c8a-4258-a9d7-3e56ab58b0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:03.719426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1158/2303] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:54:03 2022] DELETE /identity/v3/projects/b67f4759ef6a47b888079e98fee54fa3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:54:03.725160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.725560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.725930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.744545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:03.745791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:03.810836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1146/2304] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:03.876461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.876752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.877109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.896974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:03.898220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:03.907066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1159/2305] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:03.910888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:03.925983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d00140-ff98-4ff5-a3f6-fdc302f1b34f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:03.932015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:03.932688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.932978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.933260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.934190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:54:03.934507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:03.936141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:03.949987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1147/2306] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:54:03.954923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:03.964604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fd9f5-6a09-4937-b767-d929ea8d2111 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:03.969386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:03.970142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:03.970363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:03.970638 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:03.971333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:03.973454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.009101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1160/2307] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:54:03 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:54:04.010260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:04.022336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98e1a3-451a-498a-84ce-0dafef76ffa7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:04.089417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:04.090091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.090378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.090657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.091384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:04.093324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.111349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:04.119251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1148/2308] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:54:04 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:54:04.122422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:04.134918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495535f4-b571-4884-aa3c-16848f592c8a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:04.139847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:04.140513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.140804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.141081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.141804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:04.143510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.156168 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1161/2309] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:54:04.160361 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:04.170498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bc37f-f652-4ab0-a672-ad63ac227f45 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:04.175151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:04.175850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.176130 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.176404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.177122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:04.178764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbc0c6-7a9d-4f37-939d-700fd8c85562 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.190659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1149/2310] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:54:04.195379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:04.206068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336d6e3-e7b7-46f6-9054-2d5f67b47e9f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:04.211503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:04.212400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.212684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.212963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] PATH_INFO: `/v3/projects/647f4b2caa2649c094be9d09d19a0129/users/02bfd8f0a4234ffa8c5d098ef04da5ab/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.229057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647f4b2caa2649c094be9d09d19a0129, user_id=02bfd8f0a4234ffa8c5d098ef04da5ab, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:04.232177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.236684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:04.260765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1162/2311] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:54:04 2022] PUT /identity/v3/projects/647f4b2caa2649c094be9d09d19a0129/users/02bfd8f0a4234ffa8c5d098ef04da5ab/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:54:04.272151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.272440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.272719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.299911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] MFA Rules not processed for user `02bfd8f0a4234ffa8c5d098ef04da5ab`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:04.301727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:04.369701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1150/2312] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:04.380890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:04.382632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:04.464629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc55c-ce6b-40f0-b8e2-fe0c146bfdc7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:04.468407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:04.469053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:04.469331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:04.469602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:04.479830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:04.481763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28aeefd-53dd-43cf-86ee-75483f4080aa service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:04.492836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1163/2313] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:54:14.365806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=37 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:14.488784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=37 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:23.802914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:23.870716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02eafa-e43e-4c0f-8cf0-8541f22805dd None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:23.875681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:23.876568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:23.876861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:23.877139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] PATH_INFO: `/v3/users/02bfd8f0a4234ffa8c5d098ef04da5ab` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:23.878416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bfd8f0a4234ffa8c5d098ef04da5ab)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:23.881143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:23.935667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bfd8f0a4234ffa8c5d098ef04da5ab'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:23.947811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bfd8f0a4234ffa8c5d098ef04da5ab'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:23.948479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bfd8f0a4234ffa8c5d098ef04da5ab'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:23.949090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bfd8f0a4234ffa8c5d098ef04da5ab'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:23.956286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bfd8f0a4234ffa8c5d098ef04da5ab'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:23.968708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1151/2314] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:54:23 2022] DELETE /identity/v3/users/02bfd8f0a4234ffa8c5d098ef04da5ab => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:54:23.973455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:23.974003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:23.974102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:23.992000 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:23.993190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:24.049718 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1164/2315] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:54:24.060033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:24.061447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:24.118006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdccfd-34a3-494f-ac77-e6dfafcc0b3f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:24.121605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:24.122278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:24.122562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:24.122837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:24.131689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:24.133131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e913a0a-2013-414a-9032-e8aa04049b7c service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:24.142462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1152/2316] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:54:24.412821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:24.414314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:24.426267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc3db57-1616-4de4-bdfa-44bbd6ba21a4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:24.431281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:24.432154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:24.432440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:24.432715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] PATH_INFO: `/v3/projects/647f4b2caa2649c094be9d09d19a0129` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:24.433911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=647f4b2caa2649c094be9d09d19a0129)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:24.435939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:24.492407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647f4b2caa2649c094be9d09d19a0129', 'request_id': 'req-28688215-1ce4-4078-becd-da9c54108d2e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:24.500418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647f4b2caa2649c094be9d09d19a0129', 'request_id': 'req-28688215-1ce4-4078-becd-da9c54108d2e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:24.509238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647f4b2caa2649c094be9d09d19a0129', 'request_id': 'req-28688215-1ce4-4078-becd-da9c54108d2e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:24.520915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] The token cache is being invalidate because project 647f4b2caa2649c094be9d09d19a0129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:54:24.521200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:24.522447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:54:24.524332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1165/2317] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:54:24 2022] DELETE /identity/v3/projects/647f4b2caa2649c094be9d09d19a0129 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:54:26.534063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:26.534483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:26.534631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:26.563916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:26.584921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:26.586162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:26.649021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1153/2318] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 10:54:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:26.661082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:26.662607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:26.753846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688215-1ce4-4078-becd-da9c54108d2e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:26.759726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:26.760312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:26.760600 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:26.760886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:26.765871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:26.770986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:26.772930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec3ffd-8206-4762-b11c-d243ad520dc7 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:26.782142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1166/2319] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:54:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:54:36.645518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=30 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:36.779226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:43.261875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:54:43.263333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:43.358475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3002a-d4ca-456d-b867-6af8d8b28e0f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:54:43.363481 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:54:43.364122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:43.364410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:43.364689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:43.366537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:43.378157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:54:43.379751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af534d-47b7-47e8-bfe5-5c2d96a0bd25 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:54:43.390024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1154/2320] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:54:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:54:45.168219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d509-3610-4f3a-908a-bc004c7afd42 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:54:45.168740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d509-3610-4f3a-908a-bc004c7afd42 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:54:45.168849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3d509-3610-4f3a-908a-bc004c7afd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:54:45.186121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5dd3d509-3610-4f3a-908a-bc004c7afd42 None None] MFA Rules not processed for user `06f644c1c44d4dafacbd5bbdde323ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:54:45.187333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5dd3d509-3610-4f3a-908a-bc004c7afd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:54:45.241586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1167/2321] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:54:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:54:53.388435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=41 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:54:55.234587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=25 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:55:02.385204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93700-eebc-4cc4-838b-e3ed7c89e762 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:55:02.385693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93700-eebc-4cc4-838b-e3ed7c89e762 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:55:02.385878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93700-eebc-4cc4-838b-e3ed7c89e762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:55:02.404915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-e2d93700-eebc-4cc4-838b-e3ed7c89e762 None None] MFA Rules not processed for user `8e98c8ceac6b47079423b45685c5ba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:55:02.406150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e2d93700-eebc-4cc4-838b-e3ed7c89e762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:55:02.462873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1155/2322] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:55:12.455494 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=25 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:55:18.123523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a59b-21a3-4c77-a295-16206aa8fb86 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:55:18.124040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a59b-21a3-4c77-a295-16206aa8fb86 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:55:18.124147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496a59b-21a3-4c77-a295-16206aa8fb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:55:18.142308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a496a59b-21a3-4c77-a295-16206aa8fb86 None None] MFA Rules not processed for user `7433cc7ed66f4e609332c56c936487e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:55:18.143494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a496a59b-21a3-4c77-a295-16206aa8fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:55:18.200409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1168/2323] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:55:28.192983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=25 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:55:47.625334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b43b-eefe-4bba-b331-2f650c3338fe None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:55:47.625894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b43b-eefe-4bba-b331-2f650c3338fe None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:55:47.625894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054b43b-eefe-4bba-b331-2f650c3338fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:55:47.643091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8054b43b-eefe-4bba-b331-2f650c3338fe None None] MFA Rules not processed for user `a1fc33c4ee0c4b30a862f6685817bdf4`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:55:47.644292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8054b43b-eefe-4bba-b331-2f650c3338fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:55:47.704867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1156/2324] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:55:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:55:57.698199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=25 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:56:09.229713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fedc-2c89-4688-9174-3eaa2ff19306 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:09.230280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fedc-2c89-4688-9174-3eaa2ff19306 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:09.230280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1fedc-2c89-4688-9174-3eaa2ff19306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:09.248685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f9e1fedc-2c89-4688-9174-3eaa2ff19306 None None] MFA Rules not processed for user `06f644c1c44d4dafacbd5bbdde323ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:56:09.249883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f9e1fedc-2c89-4688-9174-3eaa2ff19306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:09.256731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1169/2325] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:56:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:56:19.242917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:56:21.519364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:21.519943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:21.519943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:21.537423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] MFA Rules not processed for user `8e98c8ceac6b47079423b45685c5ba3e`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:56:21.538672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:21.545504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1157/2326] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:56:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:56:31.532183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:56:36.141760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:36.142267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:36.142377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:36.159773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:56:36.161611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:36.168666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1170/2327] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:56:36.214159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:36.226953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd7aae-7d3f-45a8-85cf-9467db6385b5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:36.232038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:36.232684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:36.232969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:36.233254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:36.244288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:36.245562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:36.254053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1158/2328] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:56:39.208218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:39.209747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:39.276017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58dabe-7ce3-44b4-8eea-d059296b757a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:39.279963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:39.280724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:39.281091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:39.281447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:39.283408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:39.364440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:39.366036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:39.379365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1171/2329] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:56:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:56:40.567439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:40.568948 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:40.581796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e61aea-a00b-4f40-ad5d-2147c2c3024a service swift] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:40.585750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:40.586550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:40.586932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:40.587219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:40.588991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:40.606196 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:40.608623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:40.622871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1159/2330] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:56:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:56:47.495671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:47.497039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:47.552924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9983-4c2a-484e-a6ca-54545840eddd service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:47.556501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:47.557135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:47.557417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:47.557690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:47.559450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:47.570440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:47.571859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:47.580689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1172/2331] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:56:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:56:50.216022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:50.217430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:50.231317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdb2b5-7769-4468-97be-20c125c861f7 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:50.236355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:50.236993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:50.237315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:50.237597 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:50.239416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:50.250951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:50.252537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cf515-862c-4d9f-bf64-a1f483285034 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:50.263133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1160/2332] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:56:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:56:51.661587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:51.663017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:51.704755 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:56:51.739691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8d8aa-af63-40a2-9331-54650e1b3c10 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:51.743358 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:51.744066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:51.744363 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:51.744733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:51.746751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:51.758238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:51.759658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:51.768626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1173/2333] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:56:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:56:51.973118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:51.973586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:51.973683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:51.990802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:56:51.991972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:56:51.998964 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1161/2334] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:56:52.012009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:56:52.024270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed43bf8-4154-4011-91da-04953d939acf service glance] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:56:52.030316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:56:52.030973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:56:52.031256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:56:52.031531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:56:52.040269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:56:52.041784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53853cf-3628-4abc-9812-cf69f4c41e92 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:56:52.049806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1174/2335] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:57:01.985639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=21 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:02.046563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=24 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:07.836785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:07.850805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391daf27-5ab3-4c4d-972c-ac000d2b859f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:07.860892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:07.860892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:07.860892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:07.860892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:07.861286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:07.868724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:07.870119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:07.879486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1162/2336] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:09.454136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:09.454624 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:09.454733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:09.474499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:09.475721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:09.484523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1175/2337] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 10:57:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:09.498837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:09.511975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38885c8e-1e7d-4361-98c5-c63ed45763b1 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:09.517173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:09.517837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:09.518150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:09.518440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:09.528294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:09.529726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:09.538636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1163/2338] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:57:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:57:11.596181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:11.598282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:11.619199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2e8e3-ea62-4fe3-8c9e-eeaff35f8e05 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:11.626690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:11.627758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:11.628285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:11.628776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:11.630641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:11.650376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:11.652128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:11.663959 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1176/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:17.200325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:17.214585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0b04d-a8ee-4769-bd82-c88b8b70719d service swift] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:17.222432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:17.224058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:17.224349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:17.224792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:17.226868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:17.241430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:17.243321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:17.266287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1164/2340] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:21.658212 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=9 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:27.250389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=18 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:28.820660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:28.822128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:28.924100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e52a3b-a16c-4e4d-97b5-bcb6d678f373 service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:28.929161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:28.930114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:28.930404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:28.930685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] PATH_INFO: `/v3/users/06f644c1c44d4dafacbd5bbdde323ff0` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:28.931953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f644c1c44d4dafacbd5bbdde323ff0)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:28.934295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:28.994815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f644c1c44d4dafacbd5bbdde323ff0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.002413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f644c1c44d4dafacbd5bbdde323ff0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.003096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f644c1c44d4dafacbd5bbdde323ff0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.003739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f644c1c44d4dafacbd5bbdde323ff0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.011734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-cb349c24-5929-4c57-b376-fdcaea93a5c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f644c1c44d4dafacbd5bbdde323ff0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.031327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1177/2341] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:28 2022] DELETE /identity/v3/users/06f644c1c44d4dafacbd5bbdde323ff0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:29.215851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.237313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88f3e-526f-4746-b579-9a9068acf7fd service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.308683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.309647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.309932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.310259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] PATH_INFO: `/v3/projects/33781de828264846b90b5cbbbeee7fe5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.311456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33781de828264846b90b5cbbbeee7fe5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.313391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.367881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33781de828264846b90b5cbbbeee7fe5', 'request_id': 'req-b1501a71-fa1e-4045-ad15-9890ce4479f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.376172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33781de828264846b90b5cbbbeee7fe5', 'request_id': 'req-b1501a71-fa1e-4045-ad15-9890ce4479f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.387129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33781de828264846b90b5cbbbeee7fe5', 'request_id': 'req-b1501a71-fa1e-4045-ad15-9890ce4479f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.401204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] The token cache is being invalidate because project 33781de828264846b90b5cbbbeee7fe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:29.401630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.402419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-b1501a71-fa1e-4045-ad15-9890ce4479f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:29.404001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1165/2342] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:29 2022] DELETE /identity/v3/projects/33781de828264846b90b5cbbbeee7fe5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:29.409047 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.409344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.409625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.427564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:29.428768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:29.502162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1178/2343] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:29.510584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.510780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.511068 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.538714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:29.539954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:29.562677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1166/2344] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:29.563679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.569970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d8952-b0ee-458e-a45b-ce7d92ec7689 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.574944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.575660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.575957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.576209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.576944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:29.577254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.578944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.593155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1179/2345] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:57:29.597135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.606826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01469b1-a530-4e27-b3ac-1a4620efd0e9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.611477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.612137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.612417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.612687 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.614590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.616518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.646873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1167/2346] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:29 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:29.650862 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.661561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a987a-ade8-4424-a5c8-e98fa0752c14 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.665891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:29.727126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.727799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.728081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.728364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.729064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.731098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.756116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1180/2347] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:29 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:29.760247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.772399 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d9256-9d31-4e34-b87b-5a3b08cddb89 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.777358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.777896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.778219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.778501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.779233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.780894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.791966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1168/2348] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:29.795980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.806705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36125802-7b96-407b-94b8-2b37e3131d53 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.811559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.812209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.812493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.812769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.813487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.815102 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cefb3c-6907-4e3d-bf12-41d8f8373241 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.825571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1181/2349] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:29.829520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:29.839410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fd206-9593-42ad-a2bb-06264f14cee9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:29.844123 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:29.844956 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.845239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.845520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] PATH_INFO: `/v3/projects/be3f6c4d284943848db04d9517c97a26/users/2dfb15d4e0ea462194a6a0f2c960fb07/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.859744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3f6c4d284943848db04d9517c97a26, user_id=2dfb15d4e0ea462194a6a0f2c960fb07, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:29.862468 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:29.876552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1169/2350] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:29 2022] PUT /identity/v3/projects/be3f6c4d284943848db04d9517c97a26/users/2dfb15d4e0ea462194a6a0f2c960fb07/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:29.889088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:29.889402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:29.889705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:29.918567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] MFA Rules not processed for user `2dfb15d4e0ea462194a6a0f2c960fb07`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:29.919884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:30.000276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1182/2351] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:30.004485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.044397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:30.088945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b8296-d83d-4a86-bf4a-15d722c85471 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.094448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.095234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.095526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.095874 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.096583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.099286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.125712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1170/2352] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:30 2022] POST /identity/v3/projects => generated 363 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:30.129869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.140796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999d9e-186d-4155-8d54-bbf8f3688a4a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.199859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.200574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.200859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.201141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.201853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.203846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.226002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1183/2353] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:30 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:30.229393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.241164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067801-89a9-475f-b9e7-fc1b07a0c680 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.246393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.247298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.247592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.247877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.248616 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.250271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.261820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1171/2354] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:30.266387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.273447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:30.277669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f4b7e-24b3-4ef9-9f5b-085418cee653 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.283694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.284692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.285509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.285786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] PATH_INFO: `/v3/projects/1bb2287980be44dca28583f07f96b4be/users/ecbe75fa7bee4d628fdd69a45e5fdeb0/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.301279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb2287980be44dca28583f07f96b4be, user_id=ecbe75fa7bee4d628fdd69a45e5fdeb0, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.303996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.320157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1184/2355] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:30 2022] PUT /identity/v3/projects/1bb2287980be44dca28583f07f96b4be/users/ecbe75fa7bee4d628fdd69a45e5fdeb0/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:30.324264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.334918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544ae9-434e-4766-a00b-1444aad73ef6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.397525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.398203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.398436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.398719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.399639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.401257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c490e-e9e5-461b-a073-e8435604234b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.412672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1172/2356] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:30.417119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.427461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da412a-2312-4077-96e4-1b9984988750 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.432181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.433061 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.433348 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.433627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] PATH_INFO: `/v3/projects/1bb2287980be44dca28583f07f96b4be/users/ecbe75fa7bee4d628fdd69a45e5fdeb0/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.435652 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb2287980be44dca28583f07f96b4be, user_id=ecbe75fa7bee4d628fdd69a45e5fdeb0, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.438376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.450975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1185/2357] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:30 2022] PUT /identity/v3/projects/1bb2287980be44dca28583f07f96b4be/users/ecbe75fa7bee4d628fdd69a45e5fdeb0/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:30.464504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.464801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.465082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.493843 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] MFA Rules not processed for user `ecbe75fa7bee4d628fdd69a45e5fdeb0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:30.495108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:30.565701 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1173/2358] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:30.577766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.579689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:30.653481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adc907-b11c-4fea-9143-eaaebc8b2a51 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.658241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.658908 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.659195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.659477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.670515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.671991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.683553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1186/2359] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:30.792980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:30.804727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70775aef-04cd-4a21-8c55-c86088f737ad None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:30.808505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:30.809168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:30.809458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:30.809739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:30.839266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:30.898305 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:30.899776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f3dc3-00e3-43dd-8066-dc53611815ce service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:30.912070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1174/2360] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:33.677572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:33.741633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26861554-6b8d-4518-b5fa-dc5d7ae2e673 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:33.746215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:33.747074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:33.747356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:33.747629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] PATH_INFO: `/v3/users/2dfb15d4e0ea462194a6a0f2c960fb07` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:33.748807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dfb15d4e0ea462194a6a0f2c960fb07)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:33.750776 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:33.796628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dfb15d4e0ea462194a6a0f2c960fb07'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:33.804012 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dfb15d4e0ea462194a6a0f2c960fb07'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:33.804662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dfb15d4e0ea462194a6a0f2c960fb07'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:33.805258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dfb15d4e0ea462194a6a0f2c960fb07'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:33.812293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dfb15d4e0ea462194a6a0f2c960fb07'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:33.819340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:33.832038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1187/2361] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:33 2022] DELETE /identity/v3/users/2dfb15d4e0ea462194a6a0f2c960fb07 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:33.836410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:33.836907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:33.837017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:33.853963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:33.855161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:33.916132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1175/2362] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:33.926333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:33.927784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:33.987137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a63ef5-944f-4952-98e7-234a8e8788c6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:33.991379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:33.992020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:33.992301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:33.992573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:34.003190 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:34.004726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:34.014757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1188/2363] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:34.305089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:34.306559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:34.318358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d23b6-4f50-46df-b0e0-727c5a5aedd3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:34.322919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:34.323786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:34.324057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:34.324389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] PATH_INFO: `/v3/projects/be3f6c4d284943848db04d9517c97a26` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:34.325551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3f6c4d284943848db04d9517c97a26)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:34.327681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:34.376434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3f6c4d284943848db04d9517c97a26', 'request_id': 'req-a2be52ba-bad2-4100-a692-c62316bf78b9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.382653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3f6c4d284943848db04d9517c97a26', 'request_id': 'req-a2be52ba-bad2-4100-a692-c62316bf78b9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.388608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3f6c4d284943848db04d9517c97a26', 'request_id': 'req-a2be52ba-bad2-4100-a692-c62316bf78b9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.401586 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] The token cache is being invalidate because project be3f6c4d284943848db04d9517c97a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:34.401872 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.402519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.404060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1176/2364] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:34 2022] DELETE /identity/v3/projects/be3f6c4d284943848db04d9517c97a26 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:34.407985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:34.420136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de02d1-ad0a-40c7-931e-e655d2ffaba0 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:34.477859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:34.478756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:34.479040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:34.479319 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] PATH_INFO: `/v3/users/ecbe75fa7bee4d628fdd69a45e5fdeb0` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:34.480528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbe75fa7bee4d628fdd69a45e5fdeb0)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:34.482563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:34.529858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbe75fa7bee4d628fdd69a45e5fdeb0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.537124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbe75fa7bee4d628fdd69a45e5fdeb0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.537772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbe75fa7bee4d628fdd69a45e5fdeb0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.538401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbe75fa7bee4d628fdd69a45e5fdeb0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.545540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e112535d-8fcb-42b0-a611-2b0e5c22119a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbe75fa7bee4d628fdd69a45e5fdeb0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:34.562552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1189/2365] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:34 2022] DELETE /identity/v3/users/ecbe75fa7bee4d628fdd69a45e5fdeb0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:34.853926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:34.855532 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:34.873447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be52ba-bad2-4100-a692-c62316bf78b9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:34.921357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:34.941498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:34.942402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:34.942707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:34.942982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] PATH_INFO: `/v3/projects/1bb2287980be44dca28583f07f96b4be` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:34.944165 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb2287980be44dca28583f07f96b4be)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:34.946129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:34.997927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb2287980be44dca28583f07f96b4be', 'request_id': 'req-be86e757-96a6-4e98-9b32-c6b2bacf4282'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:35.004072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb2287980be44dca28583f07f96b4be', 'request_id': 'req-be86e757-96a6-4e98-9b32-c6b2bacf4282'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:35.009647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb2287980be44dca28583f07f96b4be', 'request_id': 'req-be86e757-96a6-4e98-9b32-c6b2bacf4282'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:35.021760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] The token cache is being invalidate because project 1bb2287980be44dca28583f07f96b4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:35.022073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:35.022684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be86e757-96a6-4e98-9b32-c6b2bacf4282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:35.024234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1177/2366] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:34 2022] DELETE /identity/v3/projects/1bb2287980be44dca28583f07f96b4be => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:35.029005 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.029459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.029553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.047157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:35.048332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:35.054018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:35.108198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1190/2367] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:35.116362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.116652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.116940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.134743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:35.135941 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:35.144279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1178/2368] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:35.149011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.159496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492b111-8bf4-4ccd-8608-fa22e533397e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.164525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.165174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.165458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.165733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.166496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:35.166915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.168517 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.181129 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1191/2369] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:57:35.185344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.194493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce549a2-0328-4402-aeef-8c3027887aa9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.199338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.199978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.200261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.200538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.201232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.203184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.240254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1179/2370] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:35.241777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.252320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd49f0-067e-44a2-a589-187d4a2570e3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.316898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.316898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.316898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.316898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.318451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.319842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.326199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.327908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:35.342738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1192/2371] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:35.343996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.354515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0418e5-4b4c-4bf7-b841-971f46c90ebf admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.359236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.359972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.360255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.360529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.361274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.362930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.375630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1193/2372] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:35.376671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.387057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8448927-3b95-451b-8567-e15a60573884 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.388171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c84cf2-273c-4e3d-af31-3c30ad670ad1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.391669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.391819 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.392306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.392450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.392604 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.392754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.392865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.393022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.393552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.394796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:35.395203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.407751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.408691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1194/2373] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:35.410643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c60769-fa70-4387-a162-bd8966318940 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.413528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.421618 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1180/2374] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:35.423063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45b617-08f4-4bf4-b31c-baa0bc81faba admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.427683 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.428564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.428848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.429124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] PATH_INFO: `/v3/projects/116a2d963af146c5a60b60dbbb3caecf/users/0fab48a6fdba420eba69f1a69fa7f7ca/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.444084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116a2d963af146c5a60b60dbbb3caecf, user_id=0fab48a6fdba420eba69f1a69fa7f7ca, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.446804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.459893 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1195/2375] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:35 2022] PUT /identity/v3/projects/116a2d963af146c5a60b60dbbb3caecf/users/0fab48a6fdba420eba69f1a69fa7f7ca/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:35.471143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.471431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.471707 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.498622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] MFA Rules not processed for user `0fab48a6fdba420eba69f1a69fa7f7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:35.499826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:35.538829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:35.568200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1181/2376] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:35.572343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.617844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:35.636300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661276f0-a9d5-4ea7-af64-04837f5f6498 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.640892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.641549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.641835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.642146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.642841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.644767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.675024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1196/2377] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:35.676070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.686924 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e5524-2882-4b50-83be-90b5aaa4d655 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.744055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.744730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.745013 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.745291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.746028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.747973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.769328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1182/2378] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:35.773977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.784627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385162f-2357-4cc3-8bec-9cdae9228555 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.789428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.790191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.790548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.790891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.791682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.793356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.807446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1197/2379] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:35.808650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.819038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c623cc3-ad20-4a69-bf16-bf0e625930b6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.823576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.824439 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.824722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.824999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] PATH_INFO: `/v3/projects/6f5587bb99d647c294c34506e88fda66/users/396fd6312b1342a29af1756fa1d48891/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.840034 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5587bb99d647c294c34506e88fda66, user_id=396fd6312b1342a29af1756fa1d48891, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.842779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.856427 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1183/2380] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:35 2022] PUT /identity/v3/projects/6f5587bb99d647c294c34506e88fda66/users/396fd6312b1342a29af1756fa1d48891/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:35.860360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.870765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431675-5e0f-4295-83e8-a51693db61b5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.928069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.928647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.928932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.929210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.929936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.931584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80939ee5-a106-4fa1-9f0d-8b2daf83e394 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.941898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1198/2381] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:35.946680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:35.956921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c21a86-0fe2-4172-9ee2-a47f21184132 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:35.961534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:35.962512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.962801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.963078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] PATH_INFO: `/v3/projects/6f5587bb99d647c294c34506e88fda66/users/396fd6312b1342a29af1756fa1d48891/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:35.965058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5587bb99d647c294c34506e88fda66, user_id=396fd6312b1342a29af1756fa1d48891, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:35.967777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:35.978209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1184/2382] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:35 2022] PUT /identity/v3/projects/6f5587bb99d647c294c34506e88fda66/users/396fd6312b1342a29af1756fa1d48891/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:35.992089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:35.992383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:35.992661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.026128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] MFA Rules not processed for user `396fd6312b1342a29af1756fa1d48891`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:36.027445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.098858 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:36.100807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.111377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1199/2383] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:36.123120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:36.124476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.167802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989053d-e580-49a2-9fd5-e64ae760365e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:36.171952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:36.172280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:36.172992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:36.173345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:36.173703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.175839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.192114 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f723aad-65a6-4118-8899-9da6668c2d84 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:36.196388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:36.197033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:36.197316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:36.197928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.207286 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:36.208745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:36.219339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1200/2384] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:36.226971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:36.253100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:36.264506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95201c8-662f-4904-be37-eacfb3fff3be service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:36.266110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:36.268007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715aeda9-91d3-41e9-9564-ab8ba7e579dc service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:36.269231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:36.270125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:36.270408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:36.270974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] PATH_INFO: `/v3/users/0fab48a6fdba420eba69f1a69fa7f7ca` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.272183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fab48a6fdba420eba69f1a69fa7f7ca)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:36.274156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:36.283007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1185/2385] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:36.332464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fab48a6fdba420eba69f1a69fa7f7ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:36.341205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fab48a6fdba420eba69f1a69fa7f7ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:36.342335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fab48a6fdba420eba69f1a69fa7f7ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:36.342664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fab48a6fdba420eba69f1a69fa7f7ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:36.353560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fab48a6fdba420eba69f1a69fa7f7ca'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:36.375011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1201/2386] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:36 2022] DELETE /identity/v3/users/0fab48a6fdba420eba69f1a69fa7f7ca => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:36.379500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:36.379906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:36.380260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.401485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:36.402915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.471770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1186/2387] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:36.492560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:36.494078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:36.565024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e78d8-0931-477a-afe0-c216adceccb3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:36.570231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:36.571016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:36.571383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:36.571729 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:36.582296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:36.584080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:36.595084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1202/2388] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:37.428895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:37.430945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:37.443761 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9f615-adbc-49df-a481-1bc93f427fa1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:37.449875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:37.451079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:37.451585 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:37.452432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] PATH_INFO: `/v3/projects/116a2d963af146c5a60b60dbbb3caecf` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:37.454135 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] RBAC: Authorizing `identity:delete_project(project_id=116a2d963af146c5a60b60dbbb3caecf)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:37.456375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:37.525234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116a2d963af146c5a60b60dbbb3caecf', 'request_id': 'req-4099e080-10f1-49f3-9967-ac2dd237625b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.532510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116a2d963af146c5a60b60dbbb3caecf', 'request_id': 'req-4099e080-10f1-49f3-9967-ac2dd237625b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.538629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116a2d963af146c5a60b60dbbb3caecf', 'request_id': 'req-4099e080-10f1-49f3-9967-ac2dd237625b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.557288 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] The token cache is being invalidate because project 116a2d963af146c5a60b60dbbb3caecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:37.557704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.558661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.560615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1187/2389] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:37 2022] DELETE /identity/v3/projects/116a2d963af146c5a60b60dbbb3caecf => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:37.564672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:37.575245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6531c8-384f-4db5-853b-1465974e98c3 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:37.655292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:37.666785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:37.667956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:37.669567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:37.669991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] PATH_INFO: `/v3/users/396fd6312b1342a29af1756fa1d48891` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:37.672362 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396fd6312b1342a29af1756fa1d48891)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:37.675096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:37.736045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396fd6312b1342a29af1756fa1d48891'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.747855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396fd6312b1342a29af1756fa1d48891'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.750508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396fd6312b1342a29af1756fa1d48891'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.751197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396fd6312b1342a29af1756fa1d48891'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.767728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f41178b2-6562-4074-b4ba-e042fa27d905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396fd6312b1342a29af1756fa1d48891'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:37.795524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1203/2390] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:37 2022] DELETE /identity/v3/users/396fd6312b1342a29af1756fa1d48891 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:38.408223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:38.409664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:38.423000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099e080-10f1-49f3-9967-ac2dd237625b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:38.500412 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:38.502200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:38.502646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:38.503141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] PATH_INFO: `/v3/projects/6f5587bb99d647c294c34506e88fda66` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:38.504599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5587bb99d647c294c34506e88fda66)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:38.508397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:38.516510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:38.572786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5587bb99d647c294c34506e88fda66', 'request_id': 'req-bbd09c39-16a9-4370-b2e0-fac68a71ddac'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:38.581354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5587bb99d647c294c34506e88fda66', 'request_id': 'req-bbd09c39-16a9-4370-b2e0-fac68a71ddac'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:38.591913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5587bb99d647c294c34506e88fda66', 'request_id': 'req-bbd09c39-16a9-4370-b2e0-fac68a71ddac'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:38.609981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] The token cache is being invalidate because project 6f5587bb99d647c294c34506e88fda66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:38.610329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:38.611105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-bbd09c39-16a9-4370-b2e0-fac68a71ddac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:38.616952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1188/2391] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:38 2022] DELETE /identity/v3/projects/6f5587bb99d647c294c34506e88fda66 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:38.618388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:38.619184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:38.619693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:38.650029 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:38.651460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:38.731667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1204/2392] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:38.740780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:38.740990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:38.741237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:38.768094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:38.769487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:38.780403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1189/2393] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:38.784443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:38.807438 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac033d-dfff-46be-ac68-d786365c2c24 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:38.815133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:38.816790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:38.816790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:38.818084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:38.818960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:38.825145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:38.827557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:38.863721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1205/2394] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:57:38.867613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:38.883429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e0c1c-6e03-4457-a007-0ffdc2b64da4 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:38.911945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:38.913908 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:38.914314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:38.914689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:38.915562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:38.918730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.007395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1190/2395] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:38 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:39.011500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.028512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448f011-7a53-4e9d-a348-0cfef332cfce admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.128735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.129566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.130703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.130988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.131687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.133659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.143713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:39.163831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1206/2396] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:39 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:39.168089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.184248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d136c-b4a4-43c5-b283-879f160087c6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.194658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.195615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.196058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.196500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.197386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.199467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.212506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1191/2397] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:39.216550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.230436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a6310-9acc-486e-bd11-136833586456 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.251871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.252602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.252894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.253587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.253923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.256488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95d33a-f300-4a4d-aadc-db5a055ff1c9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.273556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1207/2398] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:39.277879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.288906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167710c-5099-4592-8702-6d16c1ef5e09 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.296903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.297311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.297598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.297886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] PATH_INFO: `/v3/projects/0efd7c9275cf43b0b7c31325840a35b2/users/8e9c10838f2f42a1bf94679dfbfa7a69/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.315234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efd7c9275cf43b0b7c31325840a35b2, user_id=8e9c10838f2f42a1bf94679dfbfa7a69, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.318161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.337415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1192/2399] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:39 2022] PUT /identity/v3/projects/0efd7c9275cf43b0b7c31325840a35b2/users/8e9c10838f2f42a1bf94679dfbfa7a69/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:39.342827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.343332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.343803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.368258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:39.369080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:39.407516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.412630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:39.461679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1208/2400] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:39.472889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.475603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:39.503511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec224c5-bf8d-4446-a14d-b78e877a898d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.509755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.511156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.511299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.511769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.513742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:39.529773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.531541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.547502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1193/2401] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:39.548317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:39.549697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:39.564345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84d0f6-4d2e-4265-a4c8-4f55f6586cfe service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.570117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.571003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.571287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.571562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] PATH_INFO: `/v3/users/a1fc33c4ee0c4b30a862f6685817bdf4` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.573469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fc33c4ee0c4b30a862f6685817bdf4)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.575140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.576284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d94ce-ffd4-4288-bfd2-19c5ca7b0a4b None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:39.581767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:39.582536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:39.582912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:39.583310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:39.589713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:39.597074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:39.599267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:39.611942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1209/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:39.632591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fc33c4ee0c4b30a862f6685817bdf4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:39.655987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fc33c4ee0c4b30a862f6685817bdf4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:39.657274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fc33c4ee0c4b30a862f6685817bdf4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:39.658410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fc33c4ee0c4b30a862f6685817bdf4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:39.673740 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-6f63daae-6efd-4eb4-9cb6-0c20f1bd4928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fc33c4ee0c4b30a862f6685817bdf4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:39.689302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1194/2403] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:39 2022] DELETE /identity/v3/users/a1fc33c4ee0c4b30a862f6685817bdf4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:40.084016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.096361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4db07-2870-4f94-957a-dc2b9c40eb49 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.196328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.199176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.199649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.199761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] PATH_INFO: `/v3/projects/3d45f08040d949bd8949e3d1faab25c4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.203510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d45f08040d949bd8949e3d1faab25c4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.209824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:40.266662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:40.291655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d45f08040d949bd8949e3d1faab25c4', 'request_id': 'req-7f9f943e-bd16-42b3-ab95-5ffc5423c390'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:40.305931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d45f08040d949bd8949e3d1faab25c4', 'request_id': 'req-7f9f943e-bd16-42b3-ab95-5ffc5423c390'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:40.322941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d45f08040d949bd8949e3d1faab25c4', 'request_id': 'req-7f9f943e-bd16-42b3-ab95-5ffc5423c390'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:40.342970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] The token cache is being invalidate because project 3d45f08040d949bd8949e3d1faab25c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:40.343309 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:40.346381 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7f9f943e-bd16-42b3-ab95-5ffc5423c390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:40.352528 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1210/2404] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:40 2022] DELETE /identity/v3/projects/3d45f08040d949bd8949e3d1faab25c4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:40.357369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.357940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.358629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.412092 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:40.415617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.511786 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1195/2405] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:40.513532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.513855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.514192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.545416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:40.546829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.568559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1211/2406] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:40.569335 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.582563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc218b-0aa8-46a5-b558-edf4044e2055 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.595266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.596500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.597812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.598185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.603235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:40.603767 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.605983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725e1f1-c279-452f-bff6-feb49a8e22cc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:40.624154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1196/2407] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:57:40.628720 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.642082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad10e38-5f3f-4b83-8863-582f48742828 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.651750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.652524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.653987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.654334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.655806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.660763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:40.691057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.691511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.691945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.727957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1212/2408] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:40 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:40.728982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.730723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.737456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] MFA Rules not processed for user `8e9c10838f2f42a1bf94679dfbfa7a69`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:40.738683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.842531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1197/2409] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:40.846935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.849622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087121e-8447-4138-b06f-df0f72e7e15e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.859211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2b619-384a-4143-be31-01180092b663 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.860119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b619-384a-4143-be31-01180092b663 service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.860814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b619-384a-4143-be31-01180092b663 service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.861270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2b619-384a-4143-be31-01180092b663 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.862382 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:40.866596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-85c2b619-384a-4143-be31-01180092b663 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.884696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2b619-384a-4143-be31-01180092b663 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.886151 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2b619-384a-4143-be31-01180092b663 service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:40.897821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1213/2410] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:40.898864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2b619-384a-4143-be31-01180092b663 service placement] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.900963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-85c2b619-384a-4143-be31-01180092b663 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.947126 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf9481-1603-4195-a225-b93be1761d36 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.951902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.954593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.954593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.954593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.954593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.956122 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:40.984543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2b619-384a-4143-be31-01180092b663 service placement] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:40.988198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1198/2411] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:40 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:40.988198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:40.990154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:40.994254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:40.995218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:40.995573 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:40.995925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] PATH_INFO: `/v3/users/8e98c8ceac6b47079423b45685c5ba3e` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:40.997442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e98c8ceac6b47079423b45685c5ba3e)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:40.999388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.017703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:41.053812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e98c8ceac6b47079423b45685c5ba3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.057436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051282d8-94b0-4b63-a3a7-bb410f9f3cb6 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.060880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e98c8ceac6b47079423b45685c5ba3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.061182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.061550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e98c8ceac6b47079423b45685c5ba3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.061834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.062153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.062262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e98c8ceac6b47079423b45685c5ba3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.062435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.064364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.070812 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e98c8ceac6b47079423b45685c5ba3e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.078547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.079999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.089937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1199/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:41.091063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1214/2413] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:40 2022] DELETE /identity/v3/users/8e98c8ceac6b47079423b45685c5ba3e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:41.091730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.092560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.113865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101131-cc53-4872-af10-8452f184e8ac service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.184308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.185067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.185555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.185861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.186734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.188426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.199913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1200/2414] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:41.203889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.215628 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c87ef-1923-4458-8213-38b4baa47caa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.221019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.221648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.221937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.222251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.222954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.224691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.238803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1215/2415] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:41.243645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.254648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e5b0bd-4781-4f81-8878-a84b22d910a9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.261077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.261997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.262274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.262690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] PATH_INFO: `/v3/projects/a4f0b9e4c81c44f1acb872b59b7a7ba0/users/33ce280e0216416988a350cf28e764df/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.285390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f0b9e4c81c44f1acb872b59b7a7ba0, user_id=33ce280e0216416988a350cf28e764df, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.288386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e4f2-0d13-4f4c-8bcf-683cbd245fb7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.293420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.295813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.307250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1201/2416] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:41 2022] PUT /identity/v3/projects/a4f0b9e4c81c44f1acb872b59b7a7ba0/users/33ce280e0216416988a350cf28e764df/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:41.316823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee4f9e-6ec2-4c8d-84aa-c29f16f79805 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.323697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.326290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.326957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.362705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:41.366570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] MFA Rules not processed for user `33ce280e0216416988a350cf28e764df`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:41.368045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.428867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.431235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.431469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.431761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] PATH_INFO: `/v3/projects/00d4a5543a3f41a6b3dab9250c0ce9be` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.435074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d4a5543a3f41a6b3dab9250c0ce9be)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.437019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.469505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1202/2417] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:41.480651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.482076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.510466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d4a5543a3f41a6b3dab9250c0ce9be', 'request_id': 'req-9a3f79f7-7801-44f2-9943-fc9edfb444f7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.518685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d4a5543a3f41a6b3dab9250c0ce9be', 'request_id': 'req-9a3f79f7-7801-44f2-9943-fc9edfb444f7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.525266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d4a5543a3f41a6b3dab9250c0ce9be', 'request_id': 'req-9a3f79f7-7801-44f2-9943-fc9edfb444f7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.536521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] The token cache is being invalidate because project 00d4a5543a3f41a6b3dab9250c0ce9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:41.537020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.537708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9a3f79f7-7801-44f2-9943-fc9edfb444f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:41.539361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1216/2418] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:41 2022] DELETE /identity/v3/projects/00d4a5543a3f41a6b3dab9250c0ce9be => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:41.544613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80edb5-94e8-4d14-8937-27f7d2cb59f1 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.544613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80edb5-94e8-4d14-8937-27f7d2cb59f1 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.544831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80edb5-94e8-4d14-8937-27f7d2cb59f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.561329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c07d29-30af-40f7-aaec-715c5de0aa73 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.563167 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4a80edb5-94e8-4d14-8937-27f7d2cb59f1 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:41.564399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4a80edb5-94e8-4d14-8937-27f7d2cb59f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.572998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:41.626800 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.627458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.627746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.627854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1217/2419] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:41.628024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.629921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.636119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.636416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.636693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.654480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:41.655668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.664424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1218/2420] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:41.668442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.673466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:41.678238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc84e1-854d-4da7-81e3-8f0006859abe None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.683157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.683799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.684086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.684364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.685084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:41.685389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.687013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.693466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.695167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.702171 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1219/2421] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:57:41.703027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.704344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1203/2422] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:41.712744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81299b0-508e-47b5-ac7a-3f6c56f20e32 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.717351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.718015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.718304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.718581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.719260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.723304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.752142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1220/2423] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:41.757269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.768447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0012-5021-4c1b-8535-93ad8facbe23 service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.834150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.834150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.834150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.834150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.834556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.834556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.854998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1204/2424] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:41.858914 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.872426 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5b9ba-2cba-4843-9958-b98fc8a8ab99 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.878768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.879314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.879607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.879916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.880644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.882387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.894009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1221/2425] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:41.897664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.907755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc9cc2-7b3c-497a-8416-18c76e89c6ce admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.914721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.914721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.914721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.914721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.920440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.920440 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.924036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.925689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:41.933993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1205/2426] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:41.935986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:41.939241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52517c-2b84-45ed-88ef-d748fd87b5d6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.943963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.944824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.945107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.945386 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] PATH_INFO: `/v3/users/33ce280e0216416988a350cf28e764df` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.945535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf3c18-6ee3-4372-a869-be5a5d8259e1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:41.946611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ce280e0216416988a350cf28e764df)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.948594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.950159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:41.951022 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:41.951308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:41.951589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] PATH_INFO: `/v3/projects/78179b748b994323b7b990597e6d9f4a/users/5e7a80601f42441db0d43070dcc12055/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:41.970127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78179b748b994323b7b990597e6d9f4a, user_id=5e7a80601f42441db0d43070dcc12055, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:41.975647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8069e-41cc-4c86-a055-4c66242c6cf7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:41.990857 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1206/2427] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:41 2022] PUT /identity/v3/projects/78179b748b994323b7b990597e6d9f4a/users/5e7a80601f42441db0d43070dcc12055/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:42.001140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3c598-7f70-4477-a772-92537c979fab None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.001435 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3c598-7f70-4477-a772-92537c979fab None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.001715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3c598-7f70-4477-a772-92537c979fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.002427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ce280e0216416988a350cf28e764df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.009439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ce280e0216416988a350cf28e764df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.010173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ce280e0216416988a350cf28e764df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.010811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ce280e0216416988a350cf28e764df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.019516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ac767952-d9bd-45a2-968f-1bf69634a11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ce280e0216416988a350cf28e764df'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.033497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4be3c598-7f70-4477-a772-92537c979fab None None] MFA Rules not processed for user `5e7a80601f42441db0d43070dcc12055`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.034792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4be3c598-7f70-4477-a772-92537c979fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.039861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1222/2428] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:41 2022] DELETE /identity/v3/users/33ce280e0216416988a350cf28e764df => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:42.044287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.044643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.044992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.061473 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.062746 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.092565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:42.103818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1207/2429] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:42.108386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3c598-7f70-4477-a772-92537c979fab None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.109451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4be3c598-7f70-4477-a772-92537c979fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.127764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1223/2430] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:42.137228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.138626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.140254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:42.184831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3c598-7f70-4477-a772-92537c979fab None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.189535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.190326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.190684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.191037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.191734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.193714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.199109 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93b9d8-0806-4668-9dbd-098324254540 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.203191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.203838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.204125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.204405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.216187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.217890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.224059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1208/2431] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:42.225047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.228416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1224/2432] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:42.290594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6bcd6-cdf4-458d-b060-cca498a8c1cf admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.295734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.296408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.296698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.296982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.298049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.300025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.323725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1209/2433] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:42.324910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.335329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc2955-b6ee-4631-8b92-9954e936b4dc service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.340027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.340682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.340961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.341234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.342003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.343597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.354027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1225/2434] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:42.358094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.368018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37205e-edcb-4a93-b6d9-71ef8dbd549e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.372637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.375856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.375856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.375856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] PATH_INFO: `/v3/projects/b1af5beeb7f949ff9e5f5335e5571373/users/478a370627b14c04b442f137fb4da8d2/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.391234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1af5beeb7f949ff9e5f5335e5571373, user_id=478a370627b14c04b442f137fb4da8d2, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.394146 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.411070 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1210/2435] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:42 2022] PUT /identity/v3/projects/b1af5beeb7f949ff9e5f5335e5571373/users/478a370627b14c04b442f137fb4da8d2/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:42.415120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.425368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ed56-1b3b-4282-95ca-5f6221460ece admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.484876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.485550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.485829 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.486138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.486859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.488756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.507087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1226/2436] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:42.507870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.509890 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.515197 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.525110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f9bc2-3bb6-4e0c-8a97-d64377e4890c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.531017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.532096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f298c4-c1e6-45da-8f47-8bad297bfa7e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.532617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.532997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.533418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] PATH_INFO: `/v3/projects/a4f0b9e4c81c44f1acb872b59b7a7ba0` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.534753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f0b9e4c81c44f1acb872b59b7a7ba0)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.536887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.537316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.538224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.538506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.539182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] PATH_INFO: `/v3/projects/b1af5beeb7f949ff9e5f5335e5571373/users/478a370627b14c04b442f137fb4da8d2/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.541685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1af5beeb7f949ff9e5f5335e5571373, user_id=478a370627b14c04b442f137fb4da8d2, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.544552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff09e4f-75c3-4970-9ac5-f74022a66dd9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.557475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1227/2437] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:42 2022] PUT /identity/v3/projects/b1af5beeb7f949ff9e5f5335e5571373/users/478a370627b14c04b442f137fb4da8d2/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:42.558853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5257-fd85-42f4-8c97-f125adb004d0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.559209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5257-fd85-42f4-8c97-f125adb004d0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.559554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc5257-fd85-42f4-8c97-f125adb004d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.577543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-64fc5257-fd85-42f4-8c97-f125adb004d0 None None] MFA Rules not processed for user `8e9c10838f2f42a1bf94679dfbfa7a69`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.578780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-64fc5257-fd85-42f4-8c97-f125adb004d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.588460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f0b9e4c81c44f1acb872b59b7a7ba0', 'request_id': 'req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.595215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f0b9e4c81c44f1acb872b59b7a7ba0', 'request_id': 'req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.601006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f0b9e4c81c44f1acb872b59b7a7ba0', 'request_id': 'req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.604591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:42.616840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] The token cache is being invalidate because project a4f0b9e4c81c44f1acb872b59b7a7ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:42.617213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.618336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8ec86148-3bfc-442a-ae59-a4eab7bfedb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:42.620150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1211/2438] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:42 2022] DELETE /identity/v3/projects/a4f0b9e4c81c44f1acb872b59b7a7ba0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:42.623489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.623489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.623489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.624298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:42.638646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1228/2439] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:42.639918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9239-7bc3-4172-afb0-2e889cd92482 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.640206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9239-7bc3-4172-afb0-2e889cd92482 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.640484 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b9239-7bc3-4172-afb0-2e889cd92482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.650640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] MFA Rules not processed for user `478a370627b14c04b442f137fb4da8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.651842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.658878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-5a6b9239-7bc3-4172-afb0-2e889cd92482 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.660045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5a6b9239-7bc3-4172-afb0-2e889cd92482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.731034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1229/2440] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:42.734103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1212/2441] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:42.739641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.739934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.740208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.744631 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.746375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.758969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:42.760142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.768650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1230/2442] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:42.773733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.782980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08718dd5-5b09-4a1d-ae39-760d15c16987 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.788467 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.789222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.789402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.789682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.791364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:42.791364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.792410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.805999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1231/2443] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:57:42.810123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.821933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927dd4b-bd2f-4e69-b4eb-3b1e5a6a9298 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.823754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f45dd-1a2c-4c78-827d-e467a43ec133 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.826680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.827334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.827617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.827894 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.828476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.828736 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.829302 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.829785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.830194 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.830702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.841054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.842772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.853555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1213/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:42.859075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1232/2445] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:42.863665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.880477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76c682-4c6c-4801-9625-cfbff43bb4e4 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.897420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:42.900864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.958972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.959646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.959930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.960204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.960888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.963127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.968552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffcfed5-cd82-4d86-a34f-8327275769e0 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:42.972675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:42.973435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:42.973855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:42.974293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:42.976542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:42.987057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1214/2446] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:42 2022] POST /identity/v3/users => generated 282 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:42.987761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:42.989764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:42.991665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.002130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1233/2447] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:43.003265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6fbe3-f2ab-4f60-81a0-f82c85f76f96 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.006317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.008501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.009561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.010267 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.010554 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.010834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.011611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.013422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.025460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1215/2448] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:43.026692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1e032-ed5d-44bd-8666-1405e97b1430 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.029428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.031998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.032942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.033294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.033633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] PATH_INFO: `/v3/users/5e7a80601f42441db0d43070dcc12055` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.034988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7a80601f42441db0d43070dcc12055)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.035299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:43.039456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.043049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e72d0-f2c5-4563-8ee4-84a750475163 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.049930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.050650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.051459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.052300 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.053044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.055117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.070559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1216/2449] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:43.075425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.087622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec983202-0c6a-4a7d-9c13-46fa4c127b8d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.093656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.094666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.094930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.095214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] PATH_INFO: `/v3/projects/e0c0960f56544d44b2f3eff7640993f3/users/d9b1b81df510418589c03100935a2db0/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.100733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7a80601f42441db0d43070dcc12055'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.102982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:43.110956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7a80601f42441db0d43070dcc12055'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.112479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7a80601f42441db0d43070dcc12055'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.113795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7a80601f42441db0d43070dcc12055'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.113943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c0960f56544d44b2f3eff7640993f3, user_id=d9b1b81df510418589c03100935a2db0, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.118054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d76cf1-bab8-48b8-b13d-a5ca6e6fc792 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.123940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f1e8ed08-f29c-4741-9746-137b3aaa793d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7a80601f42441db0d43070dcc12055'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.133232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1217/2450] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:43 2022] PUT /identity/v3/projects/e0c0960f56544d44b2f3eff7640993f3/users/d9b1b81df510418589c03100935a2db0/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:43.144005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.144270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.144550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.146177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1234/2451] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:43 2022] DELETE /identity/v3/users/5e7a80601f42441db0d43070dcc12055 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:43.150991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.151388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.151728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.174048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:43.176170 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.178152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] MFA Rules not processed for user `d9b1b81df510418589c03100935a2db0`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:43.179472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.249621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1235/2452] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:43.262102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1218/2453] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:43.263550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.265205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.266912 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.268344 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.283343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd858c-f41c-4957-a543-ab06fa731188 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.292512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.293193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.293478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.293758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.294961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.297343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.329385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1219/2454] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:43 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:43.331915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694f026-e0a5-4490-869e-a20a8a02c44f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.333642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.394641 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.395399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.395960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.396961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.431598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aecd40-b158-4f2f-b1ef-e39df260c4f4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.437395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.438220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.438505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.439063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:43.439849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.440576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.443473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.466626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1220/2455] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:43 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:43.467868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.469633 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.470540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.481121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1236/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:43.483291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8640a-04ee-4d38-8335-5f78ab8173eb admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.488336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.489131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.489436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.489784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.490709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.492403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.503700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1221/2457] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:43.507998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.519645 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3833b-ada0-4884-b60e-0d84358fb057 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.525738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.526810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.527093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.527439 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] PATH_INFO: `/v3/projects/4cb3b923a97643869c96022aee68796e/users/0292381a9280401aacc09441c650de61/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.548402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb3b923a97643869c96022aee68796e, user_id=0292381a9280401aacc09441c650de61, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.551454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.574582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1237/2458] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:43 2022] PUT /identity/v3/projects/4cb3b923a97643869c96022aee68796e/users/0292381a9280401aacc09441c650de61/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:43.578718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.590542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb5277-91aa-4ed4-b037-b5dba63271b8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.601325 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:43.683475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.684173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.684458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.684735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.685473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.687132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c039feb6-17c3-491c-a94c-4e569b3b46d0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.701989 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1222/2459] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:43.712292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.719256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b0e91-68d8-49bb-bc07-982d657d6b93 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.728291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.729743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.730184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.730610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] PATH_INFO: `/v3/projects/4cb3b923a97643869c96022aee68796e/users/0292381a9280401aacc09441c650de61/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.733727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb3b923a97643869c96022aee68796e, user_id=0292381a9280401aacc09441c650de61, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.736583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.748206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1238/2460] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:43 2022] PUT /identity/v3/projects/4cb3b923a97643869c96022aee68796e/users/0292381a9280401aacc09441c650de61/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:43.759678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.759980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.760265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.789278 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] MFA Rules not processed for user `0292381a9280401aacc09441c650de61`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:43.790777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.826892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.828874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.877991 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1223/2461] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:43.889221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:43.890823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:43.912865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1f478-3a69-4170-b610-20bab51301b6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.918546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.919406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.919688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.919961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] PATH_INFO: `/v3/projects/78179b748b994323b7b990597e6d9f4a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.921161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78179b748b994323b7b990597e6d9f4a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.924034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:43.965777 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72ea17-8912-4535-80ef-7cdb11266b9a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:43.970963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:43.971733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:43.972032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:43.972422 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:43.987526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:43.988987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78179b748b994323b7b990597e6d9f4a', 'request_id': 'req-d7cbd6b9-800e-449c-86eb-9dde86917f89'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:43.990384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:44.000339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78179b748b994323b7b990597e6d9f4a', 'request_id': 'req-d7cbd6b9-800e-449c-86eb-9dde86917f89'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.006350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1224/2462] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:44.008356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78179b748b994323b7b990597e6d9f4a', 'request_id': 'req-d7cbd6b9-800e-449c-86eb-9dde86917f89'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.021756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] The token cache is being invalidate because project 78179b748b994323b7b990597e6d9f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:44.022237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.022964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.025352 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1239/2463] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:43 2022] DELETE /identity/v3/projects/78179b748b994323b7b990597e6d9f4a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:44.031470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:44.043699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb65d9-2bfe-4d06-a3d8-9970e3b45c24 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:44.100893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:44.136253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:44.137184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.137621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.137906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] PATH_INFO: `/v3/users/478a370627b14c04b442f137fb4da8d2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:44.143905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=478a370627b14c04b442f137fb4da8d2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:44.146359 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:44.224276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478a370627b14c04b442f137fb4da8d2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.238072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478a370627b14c04b442f137fb4da8d2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.238876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478a370627b14c04b442f137fb4da8d2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.239600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478a370627b14c04b442f137fb4da8d2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.252771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478a370627b14c04b442f137fb4da8d2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.276261 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1225/2464] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:44 2022] DELETE /identity/v3/users/478a370627b14c04b442f137fb4da8d2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:44.517556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:44.519041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:44.532602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbd6b9-800e-449c-86eb-9dde86917f89 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:44.559316 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:44.587060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:44.588725 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:44.599852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:44.600809 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.601160 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.601507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] PATH_INFO: `/v3/users/d9b1b81df510418589c03100935a2db0` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:44.602855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b1b81df510418589c03100935a2db0)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:44.604844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:44.675877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b1b81df510418589c03100935a2db0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.684247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b1b81df510418589c03100935a2db0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.689552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f56cac-f2f5-4ded-865a-5cb0361927fc admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:44.695686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b1b81df510418589c03100935a2db0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.696936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b1b81df510418589c03100935a2db0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.698691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:44.699801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.700240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.700977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] PATH_INFO: `/v3/projects/b1af5beeb7f949ff9e5f5335e5571373` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:44.702392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1af5beeb7f949ff9e5f5335e5571373)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:44.705172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:44.717761 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-bd60eadb-5099-408a-b9cb-396a625de04a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b1b81df510418589c03100935a2db0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.757145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1240/2465] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:44 2022] DELETE /identity/v3/users/d9b1b81df510418589c03100935a2db0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:44.763081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.763313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.764143 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:44.799184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1af5beeb7f949ff9e5f5335e5571373', 'request_id': 'req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.801511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:44.804399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:44.812850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1af5beeb7f949ff9e5f5335e5571373', 'request_id': 'req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.820418 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:44.824549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1af5beeb7f949ff9e5f5335e5571373', 'request_id': 'req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.851006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] The token cache is being invalidate because project b1af5beeb7f949ff9e5f5335e5571373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:44.854177 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.855426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-67ff3b30-cbb9-4f4e-88c6-9f8ea6974f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:44.861151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1226/2466] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:44 2022] DELETE /identity/v3/projects/b1af5beeb7f949ff9e5f5335e5571373 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:44.869171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aee36-4284-4bd2-81f8-b19ce2961dd3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.869171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aee36-4284-4bd2-81f8-b19ce2961dd3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.869171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aee36-4284-4bd2-81f8-b19ce2961dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:44.902225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ee6aee36-4284-4bd2-81f8-b19ce2961dd3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:44.904607 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ee6aee36-4284-4bd2-81f8-b19ce2961dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:44.950703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1241/2467] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:44.964009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:44.968529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:44.986116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1227/2468] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:44.998051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:44.998623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:44.999140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.028407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:45.030365 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:45.041446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1228/2469] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:45.045637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.053610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:45.056276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8f27-a570-42da-866f-9a07c492e99a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.060851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768f945-a43b-43ca-bea4-d0c121633af5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.061268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.061936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.062250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.063239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.063239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:45.063556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.065186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.072123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.073923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.074326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.074664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.084033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1229/2470] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:57:45.085033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.092333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.094027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.096461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16421b3-1e85-444a-8897-a334adfe5ed3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.101445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.102141 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.102428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.102704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.103389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.105528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.105651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1242/2471] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:45.138885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1230/2472] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:45 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:45.143387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.154602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30bc75-e200-4f36-b320-99023b34dd7d service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.225843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.226566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.226868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.227125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.227824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.229782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.262951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1243/2473] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:45 2022] POST /identity/v3/users => generated 303 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:45.264634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.287693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdd367-689c-41d8-8ab4-26ef0523a68c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.297121 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.298947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.299220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.300048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.300456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.302447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.316608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1231/2474] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:45.319141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.332280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b74af-a39b-4aee-a05f-a42ffd96ea4f admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.337990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.338981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.339346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.340056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.340497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.342850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.354758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1244/2475] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:45.359377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.370986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717369c-6572-40d3-b44f-ead8a29c28ad admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.376324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.377509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.378170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.378170 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] PATH_INFO: `/v3/projects/e9301ca281934b0594768cf88d6614dd/users/e7383c96c14c4ae7bcb71ee14569797f/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.403298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9301ca281934b0594768cf88d6614dd, user_id=e7383c96c14c4ae7bcb71ee14569797f, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.406222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647faf5a-f142-4ced-af7b-d1239bc63a9d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.411413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.414079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:45.426182 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1232/2476] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:45 2022] PUT /identity/v3/projects/e9301ca281934b0594768cf88d6614dd/users/e7383c96c14c4ae7bcb71ee14569797f/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:45.431793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.432080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.432351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.447223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0a78-3806-43b2-b291-e5287d0fb6aa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.465574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:45.466926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:45.516539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.518195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.518810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.518810 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] PATH_INFO: `/v3/projects/e0c0960f56544d44b2f3eff7640993f3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.520615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c0960f56544d44b2f3eff7640993f3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.522850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.540788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1233/2477] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:45.556047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.575490 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:45.599757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c0960f56544d44b2f3eff7640993f3', 'request_id': 'req-603d2584-227c-4f0c-8cb0-817efb96a6ee'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.611090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c0960f56544d44b2f3eff7640993f3', 'request_id': 'req-603d2584-227c-4f0c-8cb0-817efb96a6ee'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.623207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c0960f56544d44b2f3eff7640993f3', 'request_id': 'req-603d2584-227c-4f0c-8cb0-817efb96a6ee'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.634963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:45.639958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] The token cache is being invalidate because project e0c0960f56544d44b2f3eff7640993f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:45.640472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.641534 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.643186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1245/2478] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:45 2022] DELETE /identity/v3/projects/e0c0960f56544d44b2f3eff7640993f3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:45.647566 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:45.648700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8710c88-b0fd-4e78-bb7e-11b624bb1eb1 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.649419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:45.726481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2584-227c-4f0c-8cb0-817efb96a6ee admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:45.731331 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.732218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.732743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.732743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] PATH_INFO: `/v3/users/0292381a9280401aacc09441c650de61` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.734032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0292381a9280401aacc09441c650de61)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.735968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.740868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:45.742044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:45.742705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:45.743580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:45.746351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:45.766465 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:45.768565 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316365d8-4cc9-441f-bf22-b46ca6e3407b service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:45.781688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1234/2479] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:45.804471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0292381a9280401aacc09441c650de61'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.815013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0292381a9280401aacc09441c650de61'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.816223 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0292381a9280401aacc09441c650de61'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.817198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0292381a9280401aacc09441c650de61'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.829805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0292381a9280401aacc09441c650de61'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:45.858665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1246/2480] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:45 2022] DELETE /identity/v3/users/0292381a9280401aacc09441c650de61 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:46.462707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.464250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.464378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.464918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.465416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.497581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] MFA Rules not processed for user `e7383c96c14c4ae7bcb71ee14569797f`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:46.499157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.594056 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ec99e-1429-4a3c-a255-2dcca1ff4720 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.601637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.605731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.605731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.605731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] PATH_INFO: `/v3/projects/4cb3b923a97643869c96022aee68796e` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.607018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb3b923a97643869c96022aee68796e)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.607528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1235/2481] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:46.608583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.609288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.610268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.631933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d4463-4aba-49a4-a0cc-6a69ae698892 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.637661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.638675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.638924 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:46.639084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.639681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.640545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.642817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.677369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1236/2482] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:46.681371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.685713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb3b923a97643869c96022aee68796e', 'request_id': 'req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:46.697314 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb3b923a97643869c96022aee68796e', 'request_id': 'req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:46.705618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb3b923a97643869c96022aee68796e', 'request_id': 'req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:46.719374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] The token cache is being invalidate because project 4cb3b923a97643869c96022aee68796e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:46.719651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:46.720298 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-5c305055-fdcc-4a59-9ee1-d12fe23f67e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:46.721938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1247/2483] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:46 2022] DELETE /identity/v3/projects/4cb3b923a97643869c96022aee68796e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:46.727417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ddb64-0c2c-4950-8522-aa309e952894 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.727821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ddb64-0c2c-4950-8522-aa309e952894 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.728227 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ddb64-0c2c-4950-8522-aa309e952894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.734271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:46.750249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e73ddb64-0c2c-4950-8522-aa309e952894 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:46.751630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e73ddb64-0c2c-4950-8522-aa309e952894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.759637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c83f5-5ec5-47a7-8aaf-71b531dce8e1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.765478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.766095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1248/2484] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:46.767075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.767373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.767657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.768397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.769929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.770284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.770563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.771266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.795987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1237/2485] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:46.796505 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:46.797697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.799358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.802260 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:46.812136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1249/2486] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:46.813269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.816049 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049e751-e8c2-418e-8dbd-97d080d7ae27 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.823016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f4ab7-5db3-4454-9b1d-87d22ad30cd0 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.823695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.824753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.825050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.825515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.826590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.828493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.828691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.829291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.829581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.829863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.830707 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:46.830962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.832543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.844967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1238/2487] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:46.849382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1250/2488] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:57:46.849530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.850346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.859357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f48efd-d56c-4fea-8a9b-8704bdb556e4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.860458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83fae5-aff4-428f-bfb2-6acc0319dc91 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.866312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.867401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.867692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.867811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.868059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] PATH_INFO: `/v3/projects/9c108a72b70b4b558a77d62177fc2822/users/7812944784544e2fa78151d47517d7ba/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.868626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.868984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.869334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.870206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.873300 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.885564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c108a72b70b4b558a77d62177fc2822, user_id=7812944784544e2fa78151d47517d7ba, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.889095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.906137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1239/2489] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:46 2022] PUT /identity/v3/projects/9c108a72b70b4b558a77d62177fc2822/users/7812944784544e2fa78151d47517d7ba/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:46.906678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1251/2490] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:46.910053 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.911391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:46.979590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ca72b-dec7-41d1-8950-898a6b92be6e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.982156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be89c-90a1-49ee-bf6a-81354837327b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:46.984416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.985250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.985539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.985815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.986575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.987434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:46.988202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:46.989759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:46.989884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:46.990008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:46.990328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:46.992545 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.002120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1240/2491] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:47.004249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.015133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdf5c-6b8f-4937-8086-e1acc0652a2c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.016142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1252/2492] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:46 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:47.020003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.026161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.027376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.027816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.028237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] PATH_INFO: `/v3/projects/9c108a72b70b4b558a77d62177fc2822/users/7812944784544e2fa78151d47517d7ba/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.032396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c108a72b70b4b558a77d62177fc2822, user_id=7812944784544e2fa78151d47517d7ba, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.035560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.048995 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cda629-c9a1-4b65-9617-755527ee485e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.052223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1241/2493] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:47 2022] PUT /identity/v3/projects/9c108a72b70b4b558a77d62177fc2822/users/7812944784544e2fa78151d47517d7ba/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:47.104954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:47.137564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.138951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.140123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.141062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.145410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.147928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.162390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1253/2494] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:47.166617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.184911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa15b-c4ee-421b-be31-eea36c02ce14 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.190268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.191098 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.191329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.191710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.192926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.194193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2b63e-7d82-4e72-8597-0d51b013ab6e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.207010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1242/2495] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:47.211758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.222981 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa6199-c8f5-464d-b40c-ab67455ce6ae admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.229666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.242092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.242092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.242092 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] PATH_INFO: `/v3/projects/985a390e906545fbb729dae7c3c39d23/users/c0c6e91091f54fc68bc0cdb2b0700806/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.259784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985a390e906545fbb729dae7c3c39d23, user_id=c0c6e91091f54fc68bc0cdb2b0700806, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.262832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.279501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1254/2496] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:47 2022] PUT /identity/v3/projects/985a390e906545fbb729dae7c3c39d23/users/c0c6e91091f54fc68bc0cdb2b0700806/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:47.291105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.291105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.291457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.317448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] MFA Rules not processed for user `c0c6e91091f54fc68bc0cdb2b0700806`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:47.318672 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:47.377355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:47.390000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1243/2497] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:47.394740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.463075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d2f13-627d-4650-8532-a53fb1b4e680 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.466427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.467458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.467835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.468120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.468854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.470813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.503516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1255/2498] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:47 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:47.508367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.519732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb7713-6e47-4106-950e-fd5a0b1582e5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.579528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.580272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.580627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.580969 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.581783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.583921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.607980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1244/2499] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:47 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:47.609276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.620043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac6d79-7660-48da-9238-222acc5e8637 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.624694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.625343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.625626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.625904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.626676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.628325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.639228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1256/2500] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:47.647107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.659158 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c000f-ddbb-4251-be1d-5e8d68a1223e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.667100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.668144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.668449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.668710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] PATH_INFO: `/v3/projects/cbd9665be03448898d051bafc25bf061/users/8408a50e654a4761a441fca60005797c/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.687176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd9665be03448898d051bafc25bf061, user_id=8408a50e654a4761a441fca60005797c, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.689691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.707368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1245/2501] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:47 2022] PUT /identity/v3/projects/cbd9665be03448898d051bafc25bf061/users/8408a50e654a4761a441fca60005797c/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:47.711383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.722393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8111fe5-e51f-492c-a6c1-affc00def49a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.801310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.803935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.804413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.804855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.805779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.807714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1f162-4760-4d06-aec8-5f96c4f6bd71 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.819487 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1257/2502] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:47.823561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.834188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65936d0d-d7f9-4f97-8335-b1299d02a054 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:47.838936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:47.839822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.840227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.840515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] PATH_INFO: `/v3/projects/cbd9665be03448898d051bafc25bf061/users/8408a50e654a4761a441fca60005797c/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.842718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd9665be03448898d051bafc25bf061, user_id=8408a50e654a4761a441fca60005797c, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:47.846162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:47.859416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1246/2503] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:47 2022] PUT /identity/v3/projects/cbd9665be03448898d051bafc25bf061/users/8408a50e654a4761a441fca60005797c/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:47.871510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136695cb-a1b0-4390-96fe-57d80b2e028e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.871813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136695cb-a1b0-4390-96fe-57d80b2e028e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:47.872101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136695cb-a1b0-4390-96fe-57d80b2e028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:47.900508 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-136695cb-a1b0-4390-96fe-57d80b2e028e None None] MFA Rules not processed for user `8408a50e654a4761a441fca60005797c`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:47.901884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-136695cb-a1b0-4390-96fe-57d80b2e028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:47.921764 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:47.978524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1258/2504] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:47.990488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:47.997885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:47.999427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:47.999826 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:48.000449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:48.030876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] MFA Rules not processed for user `7812944784544e2fa78151d47517d7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:48.032730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:48.065137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35913243-aeb4-4977-bcc0-b82857e5610f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:48.069460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:48.070216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:48.070587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:48.070934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:48.083715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:48.085171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:48.096197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1247/2505] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:48.116492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1259/2506] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:57:48.178724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:48.188255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ed714-6df1-49b4-be60-f4155be07f9b service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:48.192082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:48.192739 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:48.193006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:48.193284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:48.252356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:48.266692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:48.268268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:48.278489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1248/2507] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:57:49.378797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:49.460699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7254e20-be49-47fd-a9e5-46b69c6dcdad None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:49.465488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:49.466515 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:49.466867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:49.467365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] PATH_INFO: `/v3/users/e7383c96c14c4ae7bcb71ee14569797f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:49.468757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7383c96c14c4ae7bcb71ee14569797f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:49.475689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:49.532934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:49.537572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7383c96c14c4ae7bcb71ee14569797f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:49.555213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7383c96c14c4ae7bcb71ee14569797f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:49.556117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7383c96c14c4ae7bcb71ee14569797f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:49.556878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7383c96c14c4ae7bcb71ee14569797f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:49.571589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7383c96c14c4ae7bcb71ee14569797f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:49.604881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1260/2508] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:49 2022] DELETE /identity/v3/users/e7383c96c14c4ae7bcb71ee14569797f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:49.913926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:49.928358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e9613-7f36-434e-a7a1-41cf39147aca service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:49.987978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:49.988778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:49.989060 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:49.989340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] PATH_INFO: `/v3/projects/e9301ca281934b0594768cf88d6614dd` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:49.990529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9301ca281934b0594768cf88d6614dd)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:49.992445 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.039124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9301ca281934b0594768cf88d6614dd', 'request_id': 'req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.045180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9301ca281934b0594768cf88d6614dd', 'request_id': 'req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.050592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9301ca281934b0594768cf88d6614dd', 'request_id': 'req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.062645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] The token cache is being invalidate because project e9301ca281934b0594768cf88d6614dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:50.062943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.063557 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.065114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1249/2509] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:49 2022] DELETE /identity/v3/projects/e9301ca281934b0594768cf88d6614dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:50.069757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.071192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:50.086476 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7183e8-4ad3-4dc9-9b38-2c51e1221c55 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.141658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.142551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.142831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.143103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] PATH_INFO: `/v3/users/7812944784544e2fa78151d47517d7ba` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.144262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7812944784544e2fa78151d47517d7ba)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.147063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.191728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7812944784544e2fa78151d47517d7ba'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.199304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7812944784544e2fa78151d47517d7ba'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.199950 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7812944784544e2fa78151d47517d7ba'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.200541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7812944784544e2fa78151d47517d7ba'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.207804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b1d45c82-ab44-4695-ac84-115d4597f414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7812944784544e2fa78151d47517d7ba'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.224631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1261/2510] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:50 2022] DELETE /identity/v3/users/7812944784544e2fa78151d47517d7ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:50.404284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.405999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:50.422061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a7ff1-99e3-4abc-80bc-b6517c74d39f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.479723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.480581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.480867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.481150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] PATH_INFO: `/v3/projects/9c108a72b70b4b558a77d62177fc2822` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.482366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c108a72b70b4b558a77d62177fc2822)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.484301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.505848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:50.532041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c108a72b70b4b558a77d62177fc2822', 'request_id': 'req-7d76c698-a157-4559-9f2a-8fbb41624496'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.538596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c108a72b70b4b558a77d62177fc2822', 'request_id': 'req-7d76c698-a157-4559-9f2a-8fbb41624496'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.544037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c108a72b70b4b558a77d62177fc2822', 'request_id': 'req-7d76c698-a157-4559-9f2a-8fbb41624496'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.556881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] The token cache is being invalidate because project 9c108a72b70b4b558a77d62177fc2822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:57:50.557157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.557795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d76c698-a157-4559-9f2a-8fbb41624496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:50.559387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1250/2511] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:57:50 2022] DELETE /identity/v3/projects/9c108a72b70b4b558a77d62177fc2822 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:50.564392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.564848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.564942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.582185 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:50.583384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:50.617699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:50.644356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1262/2512] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:50.652349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.652639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.652917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.671504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:50.672706 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:50.681248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1251/2513] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:50.685359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.695007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f696dc-be97-423d-ad88-64845485eb94 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.699615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.700264 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.700552 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.700833 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.701560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:57:50.701876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.703511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.714850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1263/2514] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:57:50.719026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.728954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c0c5a-4b8e-4bb0-91ee-53127739afa7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.733738 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.734480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.734774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.735051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.735789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.737702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.778469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1252/2515] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:50 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:50.783087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.794445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416de4d-b0bf-4dd7-9e96-8e4a5051055a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.862717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.863525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.863717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.864556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.865260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.867255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.892442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1264/2516] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:50 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:50.896611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.908089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac268987-d3aa-4068-ae24-e84caabd27e5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.913455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.914164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.914741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.914741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.915438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.917079 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.931753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1253/2517] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:50.933045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:50.945591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2ffb2-6807-46c0-ae86-e35259f572aa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:50.950768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:50.951658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:50.952571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:50.953086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:50.954378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:50.956503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a85b4-24a2-4cc6-9211-ab126f33659b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:50.980601 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1265/2518] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:57:50.981757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.008915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a5d1-b511-4d54-b049-81a9c08557c4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.008915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.008915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.008915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.009296 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] PATH_INFO: `/v3/projects/46b2ecd50e8e46cc94ae8bb765ed0e0a/users/a24fbdbd82c440e7ae9e5c319860bf9f/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.038228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b2ecd50e8e46cc94ae8bb765ed0e0a, user_id=a24fbdbd82c440e7ae9e5c319860bf9f, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.043582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.070798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1254/2519] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:50 2022] PUT /identity/v3/projects/46b2ecd50e8e46cc94ae8bb765ed0e0a/users/a24fbdbd82c440e7ae9e5c319860bf9f/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:51.072530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.072832 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.073127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.106795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] MFA Rules not processed for user `a24fbdbd82c440e7ae9e5c319860bf9f`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:51.108118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:51.184187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1266/2520] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:51.188730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.261753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca7108-8e09-4729-82f2-1cc9b2fb491d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.267102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.267773 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.268056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.268333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.269039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.271813 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.302287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1255/2521] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:57:51 2022] POST /identity/v3/projects => generated 357 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:57:51.303404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.314150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b611c4-ee81-47c2-9204-be0220ffaa19 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.349273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:51.389500 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.391436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.391728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.392013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.392731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.394747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.426081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1267/2522] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:57:51 2022] POST /identity/v3/users => generated 285 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:57:51.430230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.441678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39123df-d994-40fd-a64e-a020d3f6f8f8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.454722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.455429 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.455764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.456043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.456794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.459294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.473834 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1256/2523] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:51.477733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.505701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699534-ab2e-4d6f-bffb-b93b1cd746d6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.511321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.512260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.512556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.513009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] PATH_INFO: `/v3/projects/3e99c109d67a4039b3aab0013f88d457/users/9624b53319c844e387bc5b24f9663fc1/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.532691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e99c109d67a4039b3aab0013f88d457, user_id=9624b53319c844e387bc5b24f9663fc1, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.535940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.553036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1268/2524] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:51 2022] PUT /identity/v3/projects/3e99c109d67a4039b3aab0013f88d457/users/9624b53319c844e387bc5b24f9663fc1/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:57:51.557027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.567688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f884906-e140-4216-b75a-486d1145138c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.574539 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:51.650204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.650885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.651172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.651457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.652291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.653915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e563b-9f03-488b-810e-afa4c6b6281d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.664236 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1257/2525] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:57:51.668395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.679290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d4a9b-e5d6-4c27-bc1c-bb4e95bb42cb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.684350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.685252 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.685532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.685807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] PATH_INFO: `/v3/projects/3e99c109d67a4039b3aab0013f88d457/users/9624b53319c844e387bc5b24f9663fc1/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.687821 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e99c109d67a4039b3aab0013f88d457, user_id=9624b53319c844e387bc5b24f9663fc1, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.690525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.701890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1269/2526] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:57:51 2022] PUT /identity/v3/projects/3e99c109d67a4039b3aab0013f88d457/users/9624b53319c844e387bc5b24f9663fc1/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:51.713018 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.713388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.713664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.739832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] MFA Rules not processed for user `9624b53319c844e387bc5b24f9663fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:51.741039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:51.809663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1258/2527] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:51 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:51.821864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.823575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:51.882985 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a2351-c9e5-48de-982f-1d4fea8da828 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:51.886679 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:51.887324 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:51.887610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:51.887889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:51.897256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:51.898728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:51.908341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1270/2528] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:51.989238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:51.999617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974daaa3-435c-4273-92f6-9a287ff4c501 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:52.003488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:52.004209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:52.005391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:52.005512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:52.076860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:52.078508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:52.088245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1259/2529] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:51 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:54.745997 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:54.747436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:54.846690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74682126-cf28-49eb-bcd9-3d98c22bd339 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:54.855384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:54.856049 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:54.856337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:54.856617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:54.858635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:54.872035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:54.873708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:54.884768 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1271/2530] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:58.876978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:58.878498 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:58.915164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:58.944376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:58.945794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:58.946547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6f842-6439-4d69-9399-901c52977599 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:58.950162 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:58.950808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:58.951097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:58.951376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:58.953608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:58.965216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:57:59.014161 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06c51f-2630-4595-aa6b-74ccae8a2410 service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:59.017665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:59.018994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:59.019176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478de2b-1272-4666-b2d2-3d6b72d21ac3 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:59.019867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:59.020150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:59.020424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] PATH_INFO: `/v3/users/c0c6e91091f54fc68bc0cdb2b0700806` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:59.021878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c6e91091f54fc68bc0cdb2b0700806)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:59.024076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:59.029509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1260/2531] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:57:59.080408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c6e91091f54fc68bc0cdb2b0700806'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:59.092120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c6e91091f54fc68bc0cdb2b0700806'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:59.093238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c6e91091f54fc68bc0cdb2b0700806'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:59.093846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c6e91091f54fc68bc0cdb2b0700806'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:59.103511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c6e91091f54fc68bc0cdb2b0700806'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:57:59.122176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1272/2532] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:57:58 2022] DELETE /identity/v3/users/c0c6e91091f54fc68bc0cdb2b0700806 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:57:59.126746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:59.127253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:59.127708 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:59.149213 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:57:59.150492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:59.218806 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1261/2533] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:57:59.258215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:57:59.259689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:57:59.325971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca96b2-8cc6-4017-82e2-629c0611f5a7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:57:59.330121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:57:59.330815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:57:59.331098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:57:59.331387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:57:59.341433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:57:59.343162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:57:59.353602 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1273/2534] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:00.335209 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:00.337290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:00.351651 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3cb72-c423-4046-b5df-04d5a1631fb3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:00.356977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:00.358061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:00.358442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:00.358731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] PATH_INFO: `/v3/projects/985a390e906545fbb729dae7c3c39d23` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:00.359906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] RBAC: Authorizing `identity:delete_project(project_id=985a390e906545fbb729dae7c3c39d23)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:00.361829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:00.429831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985a390e906545fbb729dae7c3c39d23', 'request_id': 'req-1e57468d-1739-4b6a-be38-3b5ec23e579b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.437795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985a390e906545fbb729dae7c3c39d23', 'request_id': 'req-1e57468d-1739-4b6a-be38-3b5ec23e579b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.445850 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985a390e906545fbb729dae7c3c39d23', 'request_id': 'req-1e57468d-1739-4b6a-be38-3b5ec23e579b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.460457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] The token cache is being invalidate because project 985a390e906545fbb729dae7c3c39d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:00.460967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.461793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.463449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1262/2535] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:00 2022] DELETE /identity/v3/projects/985a390e906545fbb729dae7c3c39d23 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:00.467400 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:00.478765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2fba-dd23-4e56-9b40-07952aa8969e service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:00.518226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:00.549385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:00.550303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:00.550590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:00.550867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] PATH_INFO: `/v3/users/8408a50e654a4761a441fca60005797c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:00.552096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8408a50e654a4761a441fca60005797c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:00.554081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:00.605315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8408a50e654a4761a441fca60005797c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.612285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8408a50e654a4761a441fca60005797c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.612961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8408a50e654a4761a441fca60005797c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.613598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8408a50e654a4761a441fca60005797c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.620801 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-e1f23784-a5ca-4e02-b03d-f044d1fbd89b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8408a50e654a4761a441fca60005797c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:00.643762 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1274/2536] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:00 2022] DELETE /identity/v3/users/8408a50e654a4761a441fca60005797c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:00.981475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:00.983091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:00.996486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:01.004758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57468d-1739-4b6a-be38-3b5ec23e579b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.083340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.084393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.086164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.086164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] PATH_INFO: `/v3/projects/cbd9665be03448898d051bafc25bf061` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.087001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd9665be03448898d051bafc25bf061)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.089058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.166816 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb667f-f133-405d-987f-e81086c055c4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.167116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb667f-f133-405d-987f-e81086c055c4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.167404 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb667f-f133-405d-987f-e81086c055c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.172678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd9665be03448898d051bafc25bf061', 'request_id': 'req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:01.181367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd9665be03448898d051bafc25bf061', 'request_id': 'req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:01.186133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-73eb667f-f133-405d-987f-e81086c055c4 None None] MFA Rules not processed for user `8e9c10838f2f42a1bf94679dfbfa7a69`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:01.187349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd9665be03448898d051bafc25bf061', 'request_id': 'req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:01.187504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-73eb667f-f133-405d-987f-e81086c055c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:01.211668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] The token cache is being invalidate because project cbd9665be03448898d051bafc25bf061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:01.211970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:01.212640 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-1f9516ed-2fb3-4e03-8f50-5388a3c1182a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:01.214633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1263/2537] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:00 2022] DELETE /identity/v3/projects/cbd9665be03448898d051bafc25bf061 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:01.223350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.223881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.224457 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.243519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:01.244907 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:01.278549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1275/2538] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:01.316245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1264/2539] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:01.324621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.324922 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.325232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.350442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:01.352283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:01.366823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1276/2540] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:01.371459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.384943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a8b9f-ee6a-4522-9bc9-8092e1e9945d None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.390915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.391995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.392301 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.392587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.393328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:01.393642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.396831 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.418703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1265/2541] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:01.418903 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.430606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7da37-b09c-4523-8d8e-b2849191d511 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.439983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.440660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.440945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.441220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.444722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.451393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.489304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1277/2542] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:01.493029 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.523550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927e866-c1f7-4ba0-b218-8b0fe542fa6a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.550040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:01.597722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.600663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:01.655906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.657000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.657525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.658063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.659021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.659843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:01.662360 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.690297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1266/2543] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:01.694349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.713712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77382ab-c0ae-49ef-b161-8ef41d18976d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.716312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fe5a6-5f86-432d-9b02-92ab845791fa admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.722656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.723669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.723956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.724629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.726785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.727451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.727772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.728011 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.728915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.731237 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:01.732804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.753455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.755612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.758037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1267/2544] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:01.758962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.782008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1278/2545] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:01.796436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de62737-bdc1-4001-bc4c-320f33ff3112 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.808679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.810595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.812080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.812446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.814629 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.817720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450220f-f1e0-4ea1-a889-ff9f8f198229 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.845996 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1268/2546] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:01.846378 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:01.857716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e90c9e-d684-4cef-9c7d-f136bd0865d4 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:01.865861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:01.867083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.867594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.867937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] PATH_INFO: `/v3/projects/0b60dbd4a8a44a33bccbe9cf5556fcd7/users/2f461e25e8924cf1bc53f895de639ff5/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.888488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b60dbd4a8a44a33bccbe9cf5556fcd7, user_id=2f461e25e8924cf1bc53f895de639ff5, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:01.892410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:01.909871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1279/2547] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:01 2022] PUT /identity/v3/projects/0b60dbd4a8a44a33bccbe9cf5556fcd7/users/2f461e25e8924cf1bc53f895de639ff5/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:01.921750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:01.922238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:01.922371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:01.953989 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] MFA Rules not processed for user `2f461e25e8924cf1bc53f895de639ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:01.955205 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:02.034223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1269/2548] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:02.038164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.122556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95882ebb-f554-464d-8698-d84434f7d39c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.128748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.129502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.129791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.130174 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.130824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.132858 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.174805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1280/2549] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:02 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:02.179778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.194663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61cf57-cec7-4b55-accb-fae9518543bc None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.305562 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.306363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.306652 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.306933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.307703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.309602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.335570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1270/2550] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:02 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:02.339751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.351989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09188d3c-c688-4797-a978-9bf6c18c86d1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.358390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.358973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.359271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.359560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.360433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.362179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.374834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1281/2551] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:02.379033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.392531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a07f6-39e4-449e-883f-bf73dba3d585 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.402802 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.403776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.404066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.404351 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] PATH_INFO: `/v3/projects/9b796b16249b4020a51a5f22a7a89d73/users/c7af6c2ccbc84a63ae284a0609b17e11/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.407853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:02.434489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b796b16249b4020a51a5f22a7a89d73, user_id=c7af6c2ccbc84a63ae284a0609b17e11, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.437462 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.460097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1271/2552] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:02 2022] PUT /identity/v3/projects/9b796b16249b4020a51a5f22a7a89d73/users/c7af6c2ccbc84a63ae284a0609b17e11/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:02.463847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.475794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d37f4-6eb9-4210-8aed-17839337b12d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.544380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.545117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.545405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.545684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.546615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.548385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932986a-3cff-432a-a5c3-fb63452dd08d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.564209 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1282/2553] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:02.565424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.579043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e779c-d21c-4f4a-b192-dd97a5bf1e41 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.584891 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.585789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.586357 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.586636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] PATH_INFO: `/v3/projects/9b796b16249b4020a51a5f22a7a89d73/users/c7af6c2ccbc84a63ae284a0609b17e11/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.588803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b796b16249b4020a51a5f22a7a89d73, user_id=c7af6c2ccbc84a63ae284a0609b17e11, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.591882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.606681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1272/2554] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:02 2022] PUT /identity/v3/projects/9b796b16249b4020a51a5f22a7a89d73/users/c7af6c2ccbc84a63ae284a0609b17e11/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:02.615055 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546468b-7867-427c-b928-9d249e04b363 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.615827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546468b-7867-427c-b928-9d249e04b363 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.616123 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546468b-7867-427c-b928-9d249e04b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.669090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2546468b-7867-427c-b928-9d249e04b363 None None] MFA Rules not processed for user `c7af6c2ccbc84a63ae284a0609b17e11`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:02.670446 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2546468b-7867-427c-b928-9d249e04b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:02.685018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:02.793025 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1283/2555] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:02.802888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:02.805231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:02.935408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a094e8-7418-49f9-91cb-a0323e33624f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:02.950055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:02.952988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:02.953574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:02.954495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:02.972269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:02.974464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:02.997820 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1273/2556] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:03.013211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546468b-7867-427c-b928-9d249e04b363 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.016058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2546468b-7867-427c-b928-9d249e04b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.109846 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.141471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546468b-7867-427c-b928-9d249e04b363 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.150367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.151987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.152280 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.152695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] PATH_INFO: `/v3/users/7433cc7ed66f4e609332c56c936487e8` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.154417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7433cc7ed66f4e609332c56c936487e8)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.157688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.194203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ec172-c363-4032-a48b-85077ccb0ca5 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.202783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.203821 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.204100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.204497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] PATH_INFO: `/v3/users/2f461e25e8924cf1bc53f895de639ff5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.205803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f461e25e8924cf1bc53f895de639ff5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.208341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.219692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7433cc7ed66f4e609332c56c936487e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.231769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7433cc7ed66f4e609332c56c936487e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.232564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7433cc7ed66f4e609332c56c936487e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.233242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7433cc7ed66f4e609332c56c936487e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.246767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-26d2b0d0-5ed5-4520-8c63-00062601a36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7433cc7ed66f4e609332c56c936487e8'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.279279 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1284/2557] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:03 2022] DELETE /identity/v3/users/7433cc7ed66f4e609332c56c936487e8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:03.290784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f461e25e8924cf1bc53f895de639ff5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.322143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f461e25e8924cf1bc53f895de639ff5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.324446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f461e25e8924cf1bc53f895de639ff5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.325551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f461e25e8924cf1bc53f895de639ff5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.338694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f461e25e8924cf1bc53f895de639ff5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.347778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:03.363825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1274/2558] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:03 2022] DELETE /identity/v3/users/2f461e25e8924cf1bc53f895de639ff5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:03.368497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.368791 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.369071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.391089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:03.392315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.420564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:03.467413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1285/2559] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:03.477688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.479333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.541154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.543482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.554731 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0e081-611a-41a4-8263-8ae1466499f3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.566132 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.566933 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.567219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.567570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.574525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe46f-d843-434b-b470-501a89ccbc1f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.581448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.582630 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.582990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.583338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] PATH_INFO: `/v3/projects/2c669b26bf1845a6a3b35eb551ebcb98` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.584618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c669b26bf1845a6a3b35eb551ebcb98)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.586701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.591495 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.593428 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afe6a0-e367-432d-ad4d-0d13ccd8482a service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.608005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1275/2560] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:03.649382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c669b26bf1845a6a3b35eb551ebcb98', 'request_id': 'req-dd37f1af-de04-4c9b-887e-c151c68beaf0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.659993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c669b26bf1845a6a3b35eb551ebcb98', 'request_id': 'req-dd37f1af-de04-4c9b-887e-c151c68beaf0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.667727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c669b26bf1845a6a3b35eb551ebcb98', 'request_id': 'req-dd37f1af-de04-4c9b-887e-c151c68beaf0'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.680038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] The token cache is being invalidate because project 2c669b26bf1845a6a3b35eb551ebcb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:03.680627 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.681471 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dd37f1af-de04-4c9b-887e-c151c68beaf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:03.683537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1286/2561] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:03 2022] DELETE /identity/v3/projects/2c669b26bf1845a6a3b35eb551ebcb98 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:03.688271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.689161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.689919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.710785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:03.712400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.781944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1276/2562] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:03.789917 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.790191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.790472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.808752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:03.809978 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:03.821892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1287/2563] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:03.823847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.834664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f4520-f281-45d8-ac8e-96b12ed4c860 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.839918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.840571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.840852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.841204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.841936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:03.842273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.844310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.860733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1277/2564] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:03.861886 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.871093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a428f-b320-4637-8cb8-d3cc457b3a24 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.875834 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.876482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.876763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.877038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.877756 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.879757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:03.911990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1288/2565] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:03 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:03.913091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:03.923610 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758295a-bcfe-4be7-948c-421876f96a45 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:03.990578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:03.991341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:03.991703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:03.991982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:03.992704 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:03.994680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.018147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1278/2566] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:03 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:04.019434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.031044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebeffa-4208-4291-adf8-a915c5492042 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.036494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.037214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.037496 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.037771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.038634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.040294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.053851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1289/2567] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:04.055110 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.065309 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa987a3-f6f3-4899-90d7-6cc094689628 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.070475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.071129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.071410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.071685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.072433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.074178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6d387-b879-4691-a1cd-41f6f62d567e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.080598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:04.088003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1279/2568] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:04.090976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.101503 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb50a5-1db7-443a-b46f-666dd2770cc5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.106415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.107636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.107923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.108200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] PATH_INFO: `/v3/projects/e76a47ebf95d4b758c739aeccf3d0e27/users/2838e983efcf469ab1dc5ea55c8c38b5/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.123373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76a47ebf95d4b758c739aeccf3d0e27, user_id=2838e983efcf469ab1dc5ea55c8c38b5, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.126087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.140951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1290/2569] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:04 2022] PUT /identity/v3/projects/e76a47ebf95d4b758c739aeccf3d0e27/users/2838e983efcf469ab1dc5ea55c8c38b5/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:04.152976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.153270 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.153537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.182913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] MFA Rules not processed for user `2838e983efcf469ab1dc5ea55c8c38b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:04.184245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:04.198748 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.200470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:04.271904 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:04.280833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1280/2570] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:04.285148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.294598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db51a1c-4e78-42f2-9aad-62abedb1a8fe admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.297479 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cc473-1697-447d-b3c6-6f16abbaace5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.301694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.302373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.302594 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.302878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.303152 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] PATH_INFO: `/v3/projects/0b60dbd4a8a44a33bccbe9cf5556fcd7` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.303264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.303606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.303950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.304992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b60dbd4a8a44a33bccbe9cf5556fcd7)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.305232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.306992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.307216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.335419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1281/2571] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:04 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:04.340074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.365999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b60dbd4a8a44a33bccbe9cf5556fcd7', 'request_id': 'req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.376868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b60dbd4a8a44a33bccbe9cf5556fcd7', 'request_id': 'req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.389093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b60dbd4a8a44a33bccbe9cf5556fcd7', 'request_id': 'req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.409140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569a188-78d6-4dc5-b77d-03a903155c08 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.409782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] The token cache is being invalidate because project 0b60dbd4a8a44a33bccbe9cf5556fcd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:04.410268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.410998 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.412943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1291/2572] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:04 2022] DELETE /identity/v3/projects/0b60dbd4a8a44a33bccbe9cf5556fcd7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:04.418506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.420071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:04.521859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d4a5d-62b6-490d-9d3a-21710fef00e9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.528994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.530078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.530427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.530772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] PATH_INFO: `/v3/users/c7af6c2ccbc84a63ae284a0609b17e11` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.532706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7af6c2ccbc84a63ae284a0609b17e11)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.535058 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.557088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.559145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.559145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.559145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.559145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.560994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.590147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1282/2573] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:04 2022] POST /identity/v3/users => generated 296 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:04.591349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.594431 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:04.605021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:04.610185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c5ce7-7eed-47e3-b950-11aea5c9d3f5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.618203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7af6c2ccbc84a63ae284a0609b17e11'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.622163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.622851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.623142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.623552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.624840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.626259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.637017 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7af6c2ccbc84a63ae284a0609b17e11'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.637838 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7af6c2ccbc84a63ae284a0609b17e11'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.638620 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7af6c2ccbc84a63ae284a0609b17e11'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.647582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1283/2574] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:04.648617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.649391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7af6c2ccbc84a63ae284a0609b17e11'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:04.650787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:04.664287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb999b-160e-4572-8274-19ca33a8d07a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.665361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:04.669732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.670992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.671348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.671635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] PATH_INFO: `/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca/users/4c853ef156bf45418290afc26789074d/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.683327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1292/2575] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:04 2022] DELETE /identity/v3/users/c7af6c2ccbc84a63ae284a0609b17e11 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:04.692407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=066f21c4fdff4a2cb41169c62d3b04ca, user_id=4c853ef156bf45418290afc26789074d, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.695541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.713826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1284/2576] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:04 2022] PUT /identity/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca/users/4c853ef156bf45418290afc26789074d/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:04.727344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.813368 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf64f6-5b24-4c7f-8642-692917852967 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.880172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.880856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.881150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.881423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.882175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.883792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0f71-29f5-4329-aff9-155f54180d88 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.895524 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1293/2577] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:04.900444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:04.916556 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0d5b-bebd-4b5c-9246-bfc2ff837819 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:04.923606 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:04.925242 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.926432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.926904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] PATH_INFO: `/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca/users/4c853ef156bf45418290afc26789074d/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.930658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=066f21c4fdff4a2cb41169c62d3b04ca, user_id=4c853ef156bf45418290afc26789074d, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:04.934129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:04.949534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1285/2578] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:04 2022] PUT /identity/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca/users/4c853ef156bf45418290afc26789074d/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:04.959670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:04.960073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:04.960456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:04.992222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] MFA Rules not processed for user `4c853ef156bf45418290afc26789074d`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:04.993519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.002394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.004774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.096843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1294/2579] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:05.105879 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.106591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71452f6-56e6-4763-9f90-d97709edaa95 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.107464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.111396 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.112340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.112684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.113035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] PATH_INFO: `/v3/projects/9b796b16249b4020a51a5f22a7a89d73` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.114892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b796b16249b4020a51a5f22a7a89d73)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.116909 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.139312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:05.173153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b796b16249b4020a51a5f22a7a89d73', 'request_id': 'req-7dc55b85-58a8-4327-a523-97d32746e7cb'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:05.177596 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eaedf-34f2-40af-a765-a5d3be964f20 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.180214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b796b16249b4020a51a5f22a7a89d73', 'request_id': 'req-7dc55b85-58a8-4327-a523-97d32746e7cb'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:05.185878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b796b16249b4020a51a5f22a7a89d73', 'request_id': 'req-7dc55b85-58a8-4327-a523-97d32746e7cb'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:05.202452 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] The token cache is being invalidate because project 9b796b16249b4020a51a5f22a7a89d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:05.202828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:05.203567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7dc55b85-58a8-4327-a523-97d32746e7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:05.205525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1286/2580] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:05 2022] DELETE /identity/v3/projects/9b796b16249b4020a51a5f22a7a89d73 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:05.210483 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033644-1c4d-4033-9c53-965075abc995 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.210779 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033644-1c4d-4033-9c53-965075abc995 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.211058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033644-1c4d-4033-9c53-965075abc995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.231842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-27033644-1c4d-4033-9c53-965075abc995 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:05.232898 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-27033644-1c4d-4033-9c53-965075abc995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.237800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.239159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.239900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.240584 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.243293 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.280698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:05.314566 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1287/2581] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:05.320463 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.321548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.321548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.345875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:05.347181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.352070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.353717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.356930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1288/2582] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:05.361012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.364830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1295/2583] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:05.371226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e551e-1b00-49f8-b739-c5b3ed490716 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.378534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.379339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.379691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.380456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.381066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:05.381256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.383139 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.397040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1289/2584] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:58:05.402184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.413091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7de2f-f390-4a87-a0a1-61f3a27de774 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.417843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.418547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.419111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.419111 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.419771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.421666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.444699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.446474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.453334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1296/2585] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:05.457892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.462057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a950f-b93d-4940-a5d9-219b6daff45e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.511540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.512281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.512654 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.512997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.516868 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.526660 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb279ad-e7eb-439a-a6cb-ec8d7e5fdc39 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.531586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.532290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.532583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.532899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.533617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.536205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.554162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:05.560772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1297/2586] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:05.564825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.575673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510122a-d706-4b6c-b4d4-6e29ad60dae3 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.580554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.581285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.581635 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.586520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.586520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.586520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.586828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.587355 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.596973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1298/2588] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:05.597144 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1290/2588] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:05.600724 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.614304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c8c53-a643-4935-aed8-af3713482996 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.620113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.620962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.621343 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.621622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.622422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.624675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc22d9-0d83-423a-8e7c-ee88dbeb8fb0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.648973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1299/2589] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:05.649790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.662485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d7483-8c26-4533-8b1f-35bd8fd4bff4 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.671086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.673691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.675340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.676564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] PATH_INFO: `/v3/projects/58c1ba00daa344aa8c1ea25f667c745c/users/09e7df3285624f5189f76e71ddf4f9ea/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.704962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c1ba00daa344aa8c1ea25f667c745c, user_id=09e7df3285624f5189f76e71ddf4f9ea, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.708349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:05.743945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1291/2590] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:05 2022] PUT /identity/v3/projects/58c1ba00daa344aa8c1ea25f667c745c/users/09e7df3285624f5189f76e71ddf4f9ea/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:05.759665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.759665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.759665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.786530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] MFA Rules not processed for user `09e7df3285624f5189f76e71ddf4f9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:05.788271 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:05.870389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1300/2591] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:05.875398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:05.955571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:05.961346 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63167fe-4d01-4be2-991c-e75b549d1c26 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:05.968340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:05.969264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:05.969644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:05.969939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:05.970885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:05.972847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.000373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1292/2592] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:05 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:06.004811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.015361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d983e4-a912-4011-8435-9f3b706e0b62 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.072222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.072898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.073268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.073569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.074408 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.077001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.096843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1301/2593] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:06 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:06.100781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.110914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283b9c1-8420-44e9-8257-f05a669f3af9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.116080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.116734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.117023 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.117306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.118071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.119679 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.131061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1293/2594] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:06.135035 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.144790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf29e7-87f5-4c9c-99ab-318d00f2983b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.149610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.150553 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.150841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.151118 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] PATH_INFO: `/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb/users/484de76b20e0482a96ad4b8da174ae2d/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.165304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fa6c4f239f49e2859bd2ed8ee1ddfb, user_id=484de76b20e0482a96ad4b8da174ae2d, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.167987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.180687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1302/2595] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:06 2022] PUT /identity/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb/users/484de76b20e0482a96ad4b8da174ae2d/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:06.184510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.195080 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b8123-98c2-4a00-9e65-c8977ca9890f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.252320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.252990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.253274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.253550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.255000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.256580 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86981684-474e-4b48-91eb-4e3426378189 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.266321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1294/2596] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:06.270410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.280726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc4ee1-1fa1-4ee9-bc52-586da62392b7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.285873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.286778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.287074 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.287354 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] PATH_INFO: `/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb/users/484de76b20e0482a96ad4b8da174ae2d/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.289328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7fa6c4f239f49e2859bd2ed8ee1ddfb, user_id=484de76b20e0482a96ad4b8da174ae2d, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.291996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.302962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1303/2597] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:06 2022] PUT /identity/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb/users/484de76b20e0482a96ad4b8da174ae2d/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:06.313006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.313316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.313592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.338807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] MFA Rules not processed for user `484de76b20e0482a96ad4b8da174ae2d`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:06.339999 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:06.405750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1295/2598] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:06.416933 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:06.437065 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:06.473103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a783b-3c21-4b41-a170-e52fb2d235b6 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:06.476983 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:06.477629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:06.477964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:06.478269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:06.533708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:06.535681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166965c6-3207-41f1-934a-49da9ce2ff5b service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:06.544514 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1304/2599] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:08.785801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:08.787285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:08.871502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:08.885798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a5025-72fb-4103-a4b9-7d6b5e3e4fd3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:08.890639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:08.891505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:08.891788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:08.892067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] PATH_INFO: `/v3/users/a24fbdbd82c440e7ae9e5c319860bf9f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:08.893247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24fbdbd82c440e7ae9e5c319860bf9f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:08.896033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:08.940176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24fbdbd82c440e7ae9e5c319860bf9f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:08.952131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24fbdbd82c440e7ae9e5c319860bf9f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:08.952807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24fbdbd82c440e7ae9e5c319860bf9f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:08.953410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24fbdbd82c440e7ae9e5c319860bf9f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:08.959663 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24fbdbd82c440e7ae9e5c319860bf9f'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:08.971470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1296/2600] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:08 2022] DELETE /identity/v3/users/a24fbdbd82c440e7ae9e5c319860bf9f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:08.975984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:08.976479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:08.976591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:08.994068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:08.995258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.052860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1305/2601] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:09.062303 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.063741 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.100244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.101684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.114291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3dbeb-0d3d-4d5a-8844-157609570390 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.118934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.119799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.120081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.120356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] PATH_INFO: `/v3/users/09e7df3285624f5189f76e71ddf4f9ea` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.121558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e7df3285624f5189f76e71ddf4f9ea)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.123071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204235-d6b8-415d-98a9-b4e9b194f056 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.123521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.127147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.127794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.128083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.128373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.138381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.139817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165131a0-3e01-4d09-abc9-e77862c427d5 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.150785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1297/2602] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:09.170881 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e7df3285624f5189f76e71ddf4f9ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.178952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e7df3285624f5189f76e71ddf4f9ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.179636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e7df3285624f5189f76e71ddf4f9ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.180523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e7df3285624f5189f76e71ddf4f9ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.188290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e7df3285624f5189f76e71ddf4f9ea'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.194110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:09.210636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1306/2603] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:09 2022] DELETE /identity/v3/users/09e7df3285624f5189f76e71ddf4f9ea => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:09.215066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.215482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.215754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.239485 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:09.240805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.304475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1298/2604] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:09.314173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.315686 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.376208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1337879d-aa0f-48cf-8195-5b58f180723d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.379965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.380644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.380931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.381211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.390733 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.392469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.402913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1307/2605] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:09.432294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.433900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.470871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce285de1-7519-4f95-8e23-c91159d4a4a2 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.479078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.482870 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.483173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.483461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] PATH_INFO: `/v3/projects/46b2ecd50e8e46cc94ae8bb765ed0e0a` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.484951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b2ecd50e8e46cc94ae8bb765ed0e0a)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.487178 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.501129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:09.542776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b2ecd50e8e46cc94ae8bb765ed0e0a', 'request_id': 'req-c9c3dc54-8cf5-44b0-b193-f03c685d7131'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.550347 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b2ecd50e8e46cc94ae8bb765ed0e0a', 'request_id': 'req-c9c3dc54-8cf5-44b0-b193-f03c685d7131'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.558066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b2ecd50e8e46cc94ae8bb765ed0e0a', 'request_id': 'req-c9c3dc54-8cf5-44b0-b193-f03c685d7131'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.571928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] The token cache is being invalidate because project 46b2ecd50e8e46cc94ae8bb765ed0e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:09.572594 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.573250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.574859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1299/2606] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:09 2022] DELETE /identity/v3/projects/46b2ecd50e8e46cc94ae8bb765ed0e0a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:09.578896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.590246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ae517-b9b9-4838-9ac7-fb2d936d42f7 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.658697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.659639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.659929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.660725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] PATH_INFO: `/v3/users/9624b53319c844e387bc5b24f9663fc1` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.662008 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9624b53319c844e387bc5b24f9663fc1)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.664267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.728238 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9624b53319c844e387bc5b24f9663fc1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.735489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9624b53319c844e387bc5b24f9663fc1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.736196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9624b53319c844e387bc5b24f9663fc1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.736793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9624b53319c844e387bc5b24f9663fc1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.744284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9624b53319c844e387bc5b24f9663fc1'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.764044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1308/2607] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:09 2022] DELETE /identity/v3/users/9624b53319c844e387bc5b24f9663fc1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:09.791291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.793338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.806175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3dc54-8cf5-44b0-b193-f03c685d7131 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.870945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:09.871816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:09.872100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:09.872375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] PATH_INFO: `/v3/projects/58c1ba00daa344aa8c1ea25f667c745c` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:09.873561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c1ba00daa344aa8c1ea25f667c745c)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:09.875591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:09.923526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c1ba00daa344aa8c1ea25f667c745c', 'request_id': 'req-8fd8b96b-496f-4dbf-9568-ae31556dfc42'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.930682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c1ba00daa344aa8c1ea25f667c745c', 'request_id': 'req-8fd8b96b-496f-4dbf-9568-ae31556dfc42'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.936563 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c1ba00daa344aa8c1ea25f667c745c', 'request_id': 'req-8fd8b96b-496f-4dbf-9568-ae31556dfc42'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.949733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] The token cache is being invalidate because project 58c1ba00daa344aa8c1ea25f667c745c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:09.950119 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.950677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:09.952328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1300/2608] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:09 2022] DELETE /identity/v3/projects/58c1ba00daa344aa8c1ea25f667c745c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:09.956284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:09.958374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:09.971586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d64cc3-326a-46db-9370-c6b64f83e0e4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:09.975747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:10.031555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.032432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.032712 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.032988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] PATH_INFO: `/v3/users/484de76b20e0482a96ad4b8da174ae2d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.034248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=484de76b20e0482a96ad4b8da174ae2d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.036880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.072882 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.074541 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.095662 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484de76b20e0482a96ad4b8da174ae2d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.097513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8b96b-496f-4dbf-9568-ae31556dfc42 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.103206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.104203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.104415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484de76b20e0482a96ad4b8da174ae2d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.104523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.104718 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] PATH_INFO: `/v3/projects/3e99c109d67a4039b3aab0013f88d457` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.105485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484de76b20e0482a96ad4b8da174ae2d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.105903 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e99c109d67a4039b3aab0013f88d457)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.106141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484de76b20e0482a96ad4b8da174ae2d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.107853 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.115392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-b555281b-319d-4367-9bd8-309970cae86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484de76b20e0482a96ad4b8da174ae2d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.134349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:10.134607 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1309/2609] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:09 2022] DELETE /identity/v3/users/484de76b20e0482a96ad4b8da174ae2d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:10.160489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e99c109d67a4039b3aab0013f88d457', 'request_id': 'req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.166973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e99c109d67a4039b3aab0013f88d457', 'request_id': 'req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.172711 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e99c109d67a4039b3aab0013f88d457', 'request_id': 'req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.186967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] The token cache is being invalidate because project 3e99c109d67a4039b3aab0013f88d457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:10.187259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.187928 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-49ad2e56-e974-4fb9-a161-f3f8cc7cfa15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.189531 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1301/2610] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:10 2022] DELETE /identity/v3/projects/3e99c109d67a4039b3aab0013f88d457 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:10.194969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.195261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.195541 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.214435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:10.215726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.281033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1310/2611] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:10.289118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.289415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.289698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.308780 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:10.310113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.318915 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1302/2612] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:10.323251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.333449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cf5e6-8a13-4203-9c0b-caca3f96ea74 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.338692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.339428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.339714 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.339989 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.340775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:10.341198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.342747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.354930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1311/2613] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:10.359410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.368702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6534ef-f4f1-4eee-b670-b421f5960973 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.373678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.374369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.374658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.374937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.375626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.377559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.405648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1303/2614] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:10.410288 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.421124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8d4-4e6d-4388-a866-4c267fc4ad9a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.430274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.432156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.435774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:10.493088 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.493785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.494104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.494382 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.495064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.497257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.502719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fc45e-c6b7-4492-95a3-898efea90af2 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.508106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.508809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.509093 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.509371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] PATH_INFO: `/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.510582 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7fa6c4f239f49e2859bd2ed8ee1ddfb)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.512499 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.517869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1312/2615] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:10.522126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.531547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc641fbd-26ff-4ffa-9457-7c52c3e586d2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.537023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.537677 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.537987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.538272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.538992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.540578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.551436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1313/2616] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:10.555365 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.560810 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7fa6c4f239f49e2859bd2ed8ee1ddfb', 'request_id': 'req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.565333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696540c-8009-4b33-b4e0-974023c214ca admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.567486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7fa6c4f239f49e2859bd2ed8ee1ddfb', 'request_id': 'req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.572913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7fa6c4f239f49e2859bd2ed8ee1ddfb', 'request_id': 'req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.585279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] The token cache is being invalidate because project e7fa6c4f239f49e2859bd2ed8ee1ddfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:10.585568 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.586250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-e3b8d1d3-b8f7-4de1-83b7-d554d5cdc30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:10.587811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1304/2617] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:10 2022] DELETE /identity/v3/projects/e7fa6c4f239f49e2859bd2ed8ee1ddfb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:10.592408 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddefce-4e4c-4593-a916-3e986e70c90f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.592719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddefce-4e4c-4593-a916-3e986e70c90f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.592998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdddefce-4e4c-4593-a916-3e986e70c90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.611091 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-bdddefce-4e4c-4593-a916-3e986e70c90f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:10.612299 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-bdddefce-4e4c-4593-a916-3e986e70c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.623626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.624452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.624727 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.625073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.625929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.626530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:10.628085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.639586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1314/2618] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:10.643682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.646691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.659525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a1097-d111-45e5-a1a0-851011e4e257 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.664276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.665188 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.665502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.665779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] PATH_INFO: `/v3/projects/09857fc410ad4c539830d5fa739c8f69/users/0e85ad07afb84da28ea58c6a94aaf82b/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.680825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09857fc410ad4c539830d5fa739c8f69, user_id=0e85ad07afb84da28ea58c6a94aaf82b, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.683535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7acedd-1444-475e-b1d7-fa80587ab903 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.683721 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1305/2619] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:10.692627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.693224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.694040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.698262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1315/2620] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:10 2022] PUT /identity/v3/projects/09857fc410ad4c539830d5fa739c8f69/users/0e85ad07afb84da28ea58c6a94aaf82b/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:10.710139 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.710543 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.711519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.715066 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:10.716313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.741256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] MFA Rules not processed for user `0e85ad07afb84da28ea58c6a94aaf82b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:10.742790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:10.784004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1306/2621] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:10.789876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.799807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595602fd-f431-4eeb-8ad4-71f6ee13facc None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.806692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.807478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.807833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.808369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.808964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:10.809239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:10.810050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.812283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.822510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1316/2622] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:10.825865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1307/2623] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:58:10.826785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.829906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.839073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdcf25-bad5-455a-b73c-7249000d6376 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.840404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f2e56-370f-49f1-a2d1-4f5cb67a29a4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.844208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.844971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.845327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.845682 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.846492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.847118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.848460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.848910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.849236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.849518 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.850515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.853211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.882003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1317/2624] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:10.882919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1308/2625] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:10.883564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.886659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.946613 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e4dfd-e6a6-4519-84ec-c2ef307e9001 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.949835 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b1b2e-10a5-4257-9c7e-976aeb7c6a71 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.951287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.951956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.952242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.952519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.953205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.954693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.955173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.955484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.955836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.956181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.956900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.958862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.976493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1318/2626] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:10.981807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1309/2627] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:10 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:10.981807 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.982848 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:10.990887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc8404-cd32-420d-aa27-68a6b5629150 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.992935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569f09-5b0d-439c-a0e5-a5f1f8b218ec admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:10.995600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.996243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.996549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.996812 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.997535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:10.997646 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:10.998211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:10.998499 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:10.998786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:10.999180 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:10.999501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.001077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.002001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:11.011373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1310/2628] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:11.011530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1319/2629] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:11.015658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.017228 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.025717 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f097c-c666-4360-97dd-dba8f532e02c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.026864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754bbfd-9096-4c55-84bd-645f3287ad4c admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.030338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.030973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.031256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.031403 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.031524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.032232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.032371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.032827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.032827 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] PATH_INFO: `/v3/projects/031d22b3d3944d43818dd29465154f57/users/a2beaca89a9446b382b9626206a4c203/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.034116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.047394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1311/2630] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:11.048538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d22b3d3944d43818dd29465154f57, user_id=a2beaca89a9446b382b9626206a4c203, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.051895 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.052084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.062005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8945f6a-66dc-4654-b0cc-2be7f553f109 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.065069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1320/2631] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:11 2022] PUT /identity/v3/projects/031d22b3d3944d43818dd29465154f57/users/a2beaca89a9446b382b9626206a4c203/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:11.068797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.120579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.121376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.121656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.121929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] PATH_INFO: `/v3/projects/c085ae456b46460ea86e4cf1dfd819c2/users/43568344bfa54dd7a6b2cb5fe34422d7/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.131112 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb27e-1031-47ef-a4e7-b3a629b32745 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.135896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.136567 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c085ae456b46460ea86e4cf1dfd819c2, user_id=43568344bfa54dd7a6b2cb5fe34422d7, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.136704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.136975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.137270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.138039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.139341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee42b3-9a6f-4c3f-a172-c7cec2101118 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.139615 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.150421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1321/2632] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:11.154663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.156087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1312/2633] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:11 2022] PUT /identity/v3/projects/c085ae456b46460ea86e4cf1dfd819c2/users/43568344bfa54dd7a6b2cb5fe34422d7/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:11.161493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.161784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.162097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.165807 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb93f6-6425-40c8-bda9-6c3f5ab2b616 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.178994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:11.180200 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:11.184282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:11.224896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.225789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.226130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.226417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] PATH_INFO: `/v3/projects/031d22b3d3944d43818dd29465154f57/users/a2beaca89a9446b382b9626206a4c203/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.228411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d22b3d3944d43818dd29465154f57, user_id=a2beaca89a9446b382b9626206a4c203, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.231274 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e896a-7ca7-4f15-8555-48d637b3e449 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.242411 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1322/2634] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:11 2022] PUT /identity/v3/projects/031d22b3d3944d43818dd29465154f57/users/a2beaca89a9446b382b9626206a4c203/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:11.247916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1313/2635] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:11.252132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.252424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.252702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.257467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.258964 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:11.280100 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] MFA Rules not processed for user `a2beaca89a9446b382b9626206a4c203`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:11.281356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:11.316550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a42c0a-8581-417d-94da-3039b87fe75c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.341560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:11.350935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1323/2636] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:11.364730 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.366689 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:11.369613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.370671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.370862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.371148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.434831 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13439c8-659e-40bc-bcab-942ade162f7d None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.439545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.441203 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.441636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.443477 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.443759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.444175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.452587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1314/2637] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:11.459544 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.461859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.474987 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1324/2638] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:11.862400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.874352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee974466-ff49-43ce-b6ff-dcfdbe3b04e7 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.881990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2beaca89a9446b382b9626206a4c203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '031d22b3d3944d43818dd29465154f57', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.883450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.884131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.884605 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.886004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:11.886683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.888867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.905814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1315/2639] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:11.910057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:11.919432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0f4eb-a96a-46da-8f00-1141545bc384 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:11.942532 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2beaca89a9446b382b9626206a4c203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '031d22b3d3944d43818dd29465154f57', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:11.943276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:11.943631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:11.946031 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:11.946820 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:11.948840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:11.999934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1325/2640] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:11 2022] POST /identity/v3/projects => generated 353 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:12.001504 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:12.016899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8e93c-424b-4687-a6fb-a7b2f841721c tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:12.114876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2beaca89a9446b382b9626206a4c203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '031d22b3d3944d43818dd29465154f57', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:12.115667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:12.116002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:12.116609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:12.117311 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:12.117782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:12.120392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:12.142217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1316/2641] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:12.182284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:12.195799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e0580-134d-408b-9c7f-788d18ba6d15 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:12.201295 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a2beaca89a9446b382b9626206a4c203', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '031d22b3d3944d43818dd29465154f57', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:12.202619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:12.203019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:12.203585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] PATH_INFO: `/v3/projects/f5c8fd9dfe7c47d6b184cc9ce8f9dc35` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:12.205364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f5c8fd9dfe7c47d6b184cc9ce8f9dc35)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:12.208574 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:12.243751 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:12.328897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c8fd9dfe7c47d6b184cc9ce8f9dc35', 'request_id': 'req-201b65b6-9d9e-414a-b41c-22876a51929e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.337344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c8fd9dfe7c47d6b184cc9ce8f9dc35', 'request_id': 'req-201b65b6-9d9e-414a-b41c-22876a51929e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.346549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c8fd9dfe7c47d6b184cc9ce8f9dc35', 'request_id': 'req-201b65b6-9d9e-414a-b41c-22876a51929e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.364671 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] The token cache is being invalidate because project f5c8fd9dfe7c47d6b184cc9ce8f9dc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:12.365144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.366359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-201b65b6-9d9e-414a-b41c-22876a51929e tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.368841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1326/2642] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:12 2022] DELETE /identity/v3/projects/f5c8fd9dfe7c47d6b184cc9ce8f9dc35 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:12.586071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:12.587877 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:12.658183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:12.663782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68f33f-63b1-4de0-bfa9-40a99e1da194 tempest-VolumeQuotaClassesTest-1319167275 tempest-VolumeQuotaClassesTest-1319167275-project-admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:12.742057 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:12.743004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:12.744251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:12.744611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] PATH_INFO: `/v3/users/0e85ad07afb84da28ea58c6a94aaf82b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:12.746145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e85ad07afb84da28ea58c6a94aaf82b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:12.749902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:12.812447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e85ad07afb84da28ea58c6a94aaf82b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.826889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e85ad07afb84da28ea58c6a94aaf82b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.827886 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e85ad07afb84da28ea58c6a94aaf82b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.829028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e85ad07afb84da28ea58c6a94aaf82b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.839003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e85ad07afb84da28ea58c6a94aaf82b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:12.855202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1317/2643] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:12 2022] DELETE /identity/v3/users/0e85ad07afb84da28ea58c6a94aaf82b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:12.859489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:12.859855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:12.860208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:12.879078 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:12.880429 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:12.967464 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1327/2644] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:12.975035 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:12.976326 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:13.048075 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2d34-fe3d-469f-a841-b5ce1558d172 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:13.052256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:13.052940 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:13.053263 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:13.053879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:13.064354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:13.065823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:13.078656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1318/2645] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:13.541619 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:13.543393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:13.558449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77535-0f78-4b61-91b1-4a2eb49ce7dc None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:13.564299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:13.565179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:13.565608 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:13.566059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] PATH_INFO: `/v3/projects/09857fc410ad4c539830d5fa739c8f69` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:13.568276 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09857fc410ad4c539830d5fa739c8f69)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:13.569822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:13.638921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09857fc410ad4c539830d5fa739c8f69', 'request_id': 'req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.648361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09857fc410ad4c539830d5fa739c8f69', 'request_id': 'req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.656243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09857fc410ad4c539830d5fa739c8f69', 'request_id': 'req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.673142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] The token cache is being invalidate because project 09857fc410ad4c539830d5fa739c8f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:13.673142 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.674144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.676468 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1328/2646] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:13 2022] DELETE /identity/v3/projects/09857fc410ad4c539830d5fa739c8f69 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:13.680551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:13.694493 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd75814-a2c5-43e4-ac5c-fd797ceb11a8 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:13.720936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:13.758430 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:13.759409 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:13.759702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:13.760010 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] PATH_INFO: `/v3/users/a2beaca89a9446b382b9626206a4c203` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:13.761173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2beaca89a9446b382b9626206a4c203)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:13.763179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:13.834099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2beaca89a9446b382b9626206a4c203'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.851695 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2beaca89a9446b382b9626206a4c203'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.853109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2beaca89a9446b382b9626206a4c203'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.853733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2beaca89a9446b382b9626206a4c203'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.862082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-898e31cd-ca14-4520-836a-e0228681d324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2beaca89a9446b382b9626206a4c203'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:13.880363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1319/2647] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:13 2022] DELETE /identity/v3/users/a2beaca89a9446b382b9626206a4c203 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:14.220103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.221740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:14.239571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74e5df-3122-43d1-98c9-f2df9c0d33a4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.275593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:14.315520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.316561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.316912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.317257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] PATH_INFO: `/v3/projects/031d22b3d3944d43818dd29465154f57` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.318709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=031d22b3d3944d43818dd29465154f57)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.320864 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.376521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031d22b3d3944d43818dd29465154f57', 'request_id': 'req-60d74feb-323d-446a-aded-9325243ee28e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:14.384783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031d22b3d3944d43818dd29465154f57', 'request_id': 'req-60d74feb-323d-446a-aded-9325243ee28e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:14.392242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031d22b3d3944d43818dd29465154f57', 'request_id': 'req-60d74feb-323d-446a-aded-9325243ee28e'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:14.406419 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] The token cache is being invalidate because project 031d22b3d3944d43818dd29465154f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:14.406819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:14.407792 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-60d74feb-323d-446a-aded-9325243ee28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:14.409456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1329/2648] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:14 2022] DELETE /identity/v3/projects/031d22b3d3944d43818dd29465154f57 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:14.415099 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.415645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.415645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.434765 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:14.437340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:14.529231 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1320/2649] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:14.536896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.537379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.537772 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.557853 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:14.559284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:14.568269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1330/2650] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:14.574529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.585506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9489f-060a-4dac-9532-ac0999d2a511 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.590308 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.591076 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.591476 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.591847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.592660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:14.593065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.594966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.611197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1321/2651] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:14.612754 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.622554 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df2258-f3bb-443f-ad3b-e13cf2a2a0be None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.627675 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.628450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.629376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.629740 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.630568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.632575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.668867 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1331/2652] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:14 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:14.672895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.684885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd8e9a-d190-4310-9bf8-96920fa7f839 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.727678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:14.773608 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.774186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.774573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.775454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.776392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.779988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.848854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.849185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1322/2653] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:14 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:14.873410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50272aa3-ee91-4ea9-9f65-1a346a2b9457 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.879268 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.880208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.884154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.884154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.884154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.884154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.904572 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:14.914236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1332/2654] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:14.927665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967545ee-48ad-4849-88fa-359562a35891 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:14.934113 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:14.935094 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:14.936899 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:14.939660 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:14.942888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:14.945888 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:14.980186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1323/2655] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:14.985745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:15.010302 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c35a88-24df-453e-b0a6-d172d8d75444 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:15.018054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:15.019097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.019966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.020250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] PATH_INFO: `/v3/projects/ea6bafdd99b14f8abbee0d2f9f07b14b/users/4594073ff31d4bca9b11f5a13b1b9eea/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.023330 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:15.025534 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:15.040374 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6bafdd99b14f8abbee0d2f9f07b14b, user_id=4594073ff31d4bca9b11f5a13b1b9eea, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:15.043800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49097a56-0e54-4ed5-b79b-a9d8b9fb94a8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:15.050588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b66ea-4f29-489b-87d6-b6fb8c5bde06 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:15.061136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:15.062851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.063402 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.064043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] PATH_INFO: `/v3/users/2838e983efcf469ab1dc5ea55c8c38b5` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.065705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2838e983efcf469ab1dc5ea55c8c38b5)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:15.069724 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:15.074494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1333/2656] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:14 2022] PUT /identity/v3/projects/ea6bafdd99b14f8abbee0d2f9f07b14b/users/4594073ff31d4bca9b11f5a13b1b9eea/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:15.076150 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.076275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.076649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.134278 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] MFA Rules not processed for user `4594073ff31d4bca9b11f5a13b1b9eea`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:15.135923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:15.183410 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2838e983efcf469ab1dc5ea55c8c38b5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:15.210537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2838e983efcf469ab1dc5ea55c8c38b5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:15.211393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2838e983efcf469ab1dc5ea55c8c38b5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:15.212610 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:15.213102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2838e983efcf469ab1dc5ea55c8c38b5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:15.229603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7ebdb42-833e-4aeb-bcb1-821826052a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2838e983efcf469ab1dc5ea55c8c38b5'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:15.247155 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1334/2657] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:15.279979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:15.282231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:15.289328 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1324/2658] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:15 2022] DELETE /identity/v3/users/2838e983efcf469ab1dc5ea55c8c38b5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:15.294175 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.295041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.295041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.315238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:15.317131 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:15.393632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1325/2659] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:15.402216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:15.403726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:15.425266 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeda137-3a35-4034-9e4a-43a73ce63e56 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:15.440617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:15.487564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc36ef7-336a-46a9-a087-18375a130944 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:15.493710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:15.494788 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.495167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.495538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.511374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:15.513576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:15.522289 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:15.523939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:15.524813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:15.527087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:15.529459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1326/2660] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:15.696825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:15.702076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:15.720135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1335/2661] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:16.017225 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:16.018742 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:16.039944 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb59d4c-dcdc-4a4d-862a-c08202e4d601 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:16.045389 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:16.046441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:16.053047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:16.053387 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] PATH_INFO: `/v3/projects/e76a47ebf95d4b758c739aeccf3d0e27` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:16.054892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76a47ebf95d4b758c739aeccf3d0e27)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:16.056735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:16.125784 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76a47ebf95d4b758c739aeccf3d0e27', 'request_id': 'req-cbfec5f7-e51e-4898-8030-cf5ed6e46947'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.133997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76a47ebf95d4b758c739aeccf3d0e27', 'request_id': 'req-cbfec5f7-e51e-4898-8030-cf5ed6e46947'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.142918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76a47ebf95d4b758c739aeccf3d0e27', 'request_id': 'req-cbfec5f7-e51e-4898-8030-cf5ed6e46947'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.163041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] The token cache is being invalidate because project e76a47ebf95d4b758c739aeccf3d0e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:16.163041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.164045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-cbfec5f7-e51e-4898-8030-cf5ed6e46947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.170510 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:16.172664 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1327/2662] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:16 2022] DELETE /identity/v3/projects/e76a47ebf95d4b758c739aeccf3d0e27 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:16.188980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3b3d4-29f0-4231-a12f-952eda1e59b4 service glance] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:16.287070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:16.297857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:16.299115 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:16.301177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:16.301877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] PATH_INFO: `/v3/users/4c853ef156bf45418290afc26789074d` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:16.303407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c853ef156bf45418290afc26789074d)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:16.305649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:16.380681 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c853ef156bf45418290afc26789074d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.393921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c853ef156bf45418290afc26789074d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.396122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c853ef156bf45418290afc26789074d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.397001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c853ef156bf45418290afc26789074d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.411236 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c853ef156bf45418290afc26789074d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:16.451182 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1336/2663] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:16 2022] DELETE /identity/v3/users/4c853ef156bf45418290afc26789074d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:16.936442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:16.936910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:16.937041 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:16.937393 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:16.939067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:16.993226 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] MFA Rules not processed for user `43568344bfa54dd7a6b2cb5fe34422d7`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:16.994859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:17.096960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f173257-bf9a-4ea4-ae8d-68a2235c76be admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.101475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1328/2664] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:17.106270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.107437 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.112147 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:17.113256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.113982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.114581 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] PATH_INFO: `/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.117180 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=066f21c4fdff4a2cb41169c62d3b04ca)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.121701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.132138 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1819d-63f3-4d4a-9d82-1737e7f0ccd7 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.141986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.143281 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.143647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.144790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.146019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.148698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.188694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1329/2665] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:17.193307 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.200054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '066f21c4fdff4a2cb41169c62d3b04ca', 'request_id': 'req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:17.208700 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '066f21c4fdff4a2cb41169c62d3b04ca', 'request_id': 'req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:17.221577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '066f21c4fdff4a2cb41169c62d3b04ca', 'request_id': 'req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:17.236172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] The token cache is being invalidate because project 066f21c4fdff4a2cb41169c62d3b04ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:58:17.237141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:17.237352 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:17.238929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ec8db1ba-1701-4c4f-9150-bcc666bc6cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:17.249154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1337/2666] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:58:16 2022] DELETE /identity/v3/projects/066f21c4fdff4a2cb41169c62d3b04ca => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:58:17.258665 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4509c-da9e-4d87-b9a7-d763f8763cb7 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.259905 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4509c-da9e-4d87-b9a7-d763f8763cb7 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.260603 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f4509c-da9e-4d87-b9a7-d763f8763cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.269657 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:17.285739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-14f4509c-da9e-4d87-b9a7-d763f8763cb7 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:17.288650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-14f4509c-da9e-4d87-b9a7-d763f8763cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:17.348386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506b49c-2135-4a80-9519-fe65233a473f admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.358456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.359555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.360036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.360666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.361696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.365377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.398375 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1330/2667] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:17.402353 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.405636 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:17.409256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1338/2668] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:17.417385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.417840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.422166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.437219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83214b-ea9c-4aec-9346-cba9c0feb46b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.445646 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.447219 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.447844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.448291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.451710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.453455 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.459937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:17.462335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:17.475095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1331/2669] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:17.475095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.476098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1339/2670] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:17.484530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.506715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6866c-9bf7-434c-b2aa-85ef9a3fca63 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.516219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.517410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.517410 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.517634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.518441 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:58:17.519015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.521063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.524266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7f48d-ec3d-4501-8ce9-e02a8cd10a5c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.532578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.533118 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.533414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.534082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] PATH_INFO: `/v3/projects/3b3bfe966a0344d2847514ec296dee76/users/1a18cd72a1334aad88e3bea937084262/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.542244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1340/2671] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:58:17.543359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.557865 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3bfe966a0344d2847514ec296dee76, user_id=1a18cd72a1334aad88e3bea937084262, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.562994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.570849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f86b1-4155-4592-ae88-64f303cbdedb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.580285 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.581106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.581494 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.581907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.582492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1332/2672] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:17 2022] PUT /identity/v3/projects/3b3bfe966a0344d2847514ec296dee76/users/1a18cd72a1334aad88e3bea937084262/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:17.585516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.585516 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.587727 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.624846 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1341/2673] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:17.626080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.679847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240058a-fc09-4ddb-91c1-76244f818ba8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.702218 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4915ce-bb76-404f-9627-10eca8afdb23 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.709384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.710557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.710849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.711244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.711957 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.713961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.739783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1342/2674] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:17.742637 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.750043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.750814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.751104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.751380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.752128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.754735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4848bd-ee5d-4991-82f8-b2dc662251cc admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.754892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.759222 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.759796 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.760082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.760357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.761079 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.762721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.769805 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1333/2675] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:17.771464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.781991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1343/2676] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:17.787168 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4dff0-6209-4cfa-9b6d-0d45126469ea admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.787850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.791980 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.792867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.793277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.793602 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] PATH_INFO: `/v3/projects/3b3bfe966a0344d2847514ec296dee76/users/1a18cd72a1334aad88e3bea937084262/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.796092 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3bfe966a0344d2847514ec296dee76, user_id=1a18cd72a1334aad88e3bea937084262, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.798832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.798988 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c11d0-be2e-4f22-91b2-c8a9664014a2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.807854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.809345 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.809735 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.810062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.810871 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.812716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f912c0-a35b-427d-a739-3013cc6e4261 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.815154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1334/2677] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:17 2022] PUT /identity/v3/projects/3b3bfe966a0344d2847514ec296dee76/users/1a18cd72a1334aad88e3bea937084262/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:17.839884 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1344/2678] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:17.841140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:17.846520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:17.926621 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a51f43-a9c4-457e-80aa-c2e8084a329c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:17.932502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:17.933577 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.934058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.934406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] PATH_INFO: `/v3/projects/bbc4931ce6094196876f0e1cbd3f67d5/users/4cc02688ead142d982587d1659b6f4ee/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:17.952703 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc4931ce6094196876f0e1cbd3f67d5, user_id=4cc02688ead142d982587d1659b6f4ee, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:17.957345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:17.973609 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1335/2679] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:17 2022] PUT /identity/v3/projects/bbc4931ce6094196876f0e1cbd3f67d5/users/4cc02688ead142d982587d1659b6f4ee/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:17.985200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536058a-d024-478a-8d22-c08da972896c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:17.985766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536058a-d024-478a-8d22-c08da972896c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:17.986587 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536058a-d024-478a-8d22-c08da972896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.038359 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8536058a-d024-478a-8d22-c08da972896c None None] MFA Rules not processed for user `4cc02688ead142d982587d1659b6f4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:18.042547 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8536058a-d024-478a-8d22-c08da972896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:18.090192 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:18.169659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1345/2680] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:18.173992 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.302143 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94eda-985d-4b7e-ae50-31d41cd96920 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.311726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.312522 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.313240 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.313985 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.316497 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.320127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.366683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1336/2681] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:58:18 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:58:18.372250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536058a-d024-478a-8d22-c08da972896c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.384163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536058a-d024-478a-8d22-c08da972896c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.455201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.456351 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.456655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.456952 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.458068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.460054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.482958 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1346/2682] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:58:18 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:58:18.486911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.502507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a6d-0c14-41c5-8fb4-0373ea30e7a2 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.508372 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.509338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.509774 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.510304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.511391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.516461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.530391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1337/2683] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:58:18.535234 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.547340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59070734-83ac-47bd-92f5-c446e5b9d0cb admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.557533 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.558519 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.558912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.559469 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] PATH_INFO: `/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e/users/56e29b5df31b4bceac958f1f2523c36b/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.581401 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4cae215c6146ddaf1cab2fa0bc0a4e, user_id=56e29b5df31b4bceac958f1f2523c36b, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.593663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.618356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1347/2684] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:18 2022] PUT /identity/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e/users/56e29b5df31b4bceac958f1f2523c36b/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:18.623043 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.636971 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c0e6-1e8a-438c-9046-6798b439bec8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.807202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.810081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.810593 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.810736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.812185 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.818100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf4a11-e2ae-4270-8484-21e82801539e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.823134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:18.849990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1338/2685] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:58:18.852920 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:18.870595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb9f0-01c7-48c7-a7ac-e860cf326723 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:18.884096 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:18.885156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.885550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.885935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] PATH_INFO: `/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e/users/56e29b5df31b4bceac958f1f2523c36b/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.888502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4cae215c6146ddaf1cab2fa0bc0a4e, user_id=56e29b5df31b4bceac958f1f2523c36b, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:18.891559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:18.909546 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1348/2686] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:58:18 2022] PUT /identity/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e/users/56e29b5df31b4bceac958f1f2523c36b/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:18.921064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f852-514b-4066-8cb4-329d7fd15b54 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:18.923694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f852-514b-4066-8cb4-329d7fd15b54 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:18.924967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f852-514b-4066-8cb4-329d7fd15b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:18.982077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-f6d9f852-514b-4066-8cb4-329d7fd15b54 None None] MFA Rules not processed for user `56e29b5df31b4bceac958f1f2523c36b`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:18.983416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-f6d9f852-514b-4066-8cb4-329d7fd15b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:19.108186 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1339/2687] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:19.121406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:19.123655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:19.243960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e4c7-14f2-4192-af98-e4344a16bbce admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:19.263609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:19.264305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:19.264902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:19.264902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:19.310550 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:19.362577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:19.364777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:19.390116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1349/2688] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:58:23.608617 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644cdd-57d1-4bd8-9ded-6ffea1618014 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:23.609207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644cdd-57d1-4bd8-9ded-6ffea1618014 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:23.609207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644cdd-57d1-4bd8-9ded-6ffea1618014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:23.641340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-78644cdd-57d1-4bd8-9ded-6ffea1618014 None None] MFA Rules not processed for user `1a18cd72a1334aad88e3bea937084262`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:23.642804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-78644cdd-57d1-4bd8-9ded-6ffea1618014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:23.727286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1340/2689] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:23.735262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:23.738576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:23.823045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0808b4-6ca1-4f81-b83c-9643d6641b4c service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:23.828850 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:23.829631 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:23.829915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:23.830231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:23.832640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:23.896775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:23.898440 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:23.911257 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1350/2690] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:24.949306 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166396-e094-485e-98e3-70664efcee19 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:24.949804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166396-e094-485e-98e3-70664efcee19 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:24.949913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47166396-e094-485e-98e3-70664efcee19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:24.969611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-47166396-e094-485e-98e3-70664efcee19 None None] MFA Rules not processed for user `43568344bfa54dd7a6b2cb5fe34422d7`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:24.971038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-47166396-e094-485e-98e3-70664efcee19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:24.980525 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1341/2691] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:25.629614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:25.639258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafec0e-39ef-4c0e-a90d-4ef17555eb56 service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:25.642963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:25.643609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:25.643896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:25.644173 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:25.654405 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:25.655872 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:25.665391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1351/2692] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:58:32.324735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:32.325259 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:32.325362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:32.342988 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:32.344189 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:32.401233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1342/2693] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 10:58:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:32.453267 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:32.454921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:32.510310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:32.537135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b19f08-a8a3-4feb-9f53-b6f46304d898 service cinder] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:32.542767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:32.543425 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:32.543709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:32.543990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:32.553033 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:32.554680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5deb12b-0a57-48fe-ad87-58e0d83dca01 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:32.563229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1352/2694] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:58:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 10:58:42.394472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=98 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:42.559931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=9 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:58:59.167832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:59.190744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:59.191254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:59.191373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:59.209634 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:59.211915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:59.219420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1353/2695] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:58:59.232157 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:59.249059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef463f-5cc7-4233-adc3-bb3d6860aded None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:59.251961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293270aa-7873-4f7f-921c-e89259da4dca None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:59.254033 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:59.254910 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:59.255193 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:59.255470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] PATH_INFO: `/v3/users/4cc02688ead142d982587d1659b6f4ee` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:59.256709 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc02688ead142d982587d1659b6f4ee)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:59.258760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:59.259067 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:59.260072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:59.260364 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:59.260651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:59.270551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:59.272418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166bc7a-a2d8-43db-9bdf-ea355068584f service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:59.281504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1354/2696] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 10:58:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 10:58:59.311405 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc02688ead142d982587d1659b6f4ee'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:59.328343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc02688ead142d982587d1659b6f4ee'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:59.329044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc02688ead142d982587d1659b6f4ee'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:59.329661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc02688ead142d982587d1659b6f4ee'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:59.340377 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc02688ead142d982587d1659b6f4ee'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:58:59.361081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1343/2697] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:58:59 2022] DELETE /identity/v3/users/4cc02688ead142d982587d1659b6f4ee => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:58:59.365774 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:59.366104 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:59.366389 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:59.386882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:58:59.388321 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:59.462431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1355/2698] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:58:59.472684 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:58:59.474340 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:58:59.542760 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241530c8-0fac-48eb-88da-526e44614f74 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:58:59.547215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:58:59.547961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:58:59.548327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:58:59.548772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:58:59.562555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:58:59.564125 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:58:59.575380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1344/2699] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:00.202539 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:00.204073 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:00.249166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4c43c-4758-4533-b4fc-510218394f75 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:00.254993 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:00.255929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:00.256414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:00.256693 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] PATH_INFO: `/v3/projects/bbc4931ce6094196876f0e1cbd3f67d5` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:00.258135 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc4931ce6094196876f0e1cbd3f67d5)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:00.260097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:00.314655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc4931ce6094196876f0e1cbd3f67d5', 'request_id': 'req-a7dec907-b483-42bb-b7e6-8e71f170bda7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.322328 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc4931ce6094196876f0e1cbd3f67d5', 'request_id': 'req-a7dec907-b483-42bb-b7e6-8e71f170bda7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.328555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc4931ce6094196876f0e1cbd3f67d5', 'request_id': 'req-a7dec907-b483-42bb-b7e6-8e71f170bda7'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.340294 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] The token cache is being invalidate because project bbc4931ce6094196876f0e1cbd3f67d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:00.340585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.341233 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.342813 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1356/2700] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:00 2022] DELETE /identity/v3/projects/bbc4931ce6094196876f0e1cbd3f67d5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:00.346844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:00.358369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac6907-9596-469f-bda7-9df8dfbadfb1 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:00.377051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:00.430549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:00.431634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:00.432128 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:00.432622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] PATH_INFO: `/v3/users/56e29b5df31b4bceac958f1f2523c36b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:00.434108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e29b5df31b4bceac958f1f2523c36b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:00.436228 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:00.490349 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e29b5df31b4bceac958f1f2523c36b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.506842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e29b5df31b4bceac958f1f2523c36b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.508276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e29b5df31b4bceac958f1f2523c36b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.509145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e29b5df31b4bceac958f1f2523c36b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.516477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-0ebc971c-3a96-4545-86c8-e0b9a97dee26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e29b5df31b4bceac958f1f2523c36b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:00.529214 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1345/2701] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:00 2022] DELETE /identity/v3/users/56e29b5df31b4bceac958f1f2523c36b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:00.930984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:00.932448 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:00.945773 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dec907-b483-42bb-b7e6-8e71f170bda7 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.021561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.022465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.022779 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.022966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] PATH_INFO: `/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.024509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4cae215c6146ddaf1cab2fa0bc0a4e)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.026474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.034064 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:01.087967 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4cae215c6146ddaf1cab2fa0bc0a4e', 'request_id': 'req-316f892d-c45d-4955-a177-fa131f72640d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.097332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4cae215c6146ddaf1cab2fa0bc0a4e', 'request_id': 'req-316f892d-c45d-4955-a177-fa131f72640d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.106734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4cae215c6146ddaf1cab2fa0bc0a4e', 'request_id': 'req-316f892d-c45d-4955-a177-fa131f72640d'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.117963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] The token cache is being invalidate because project 5f4cae215c6146ddaf1cab2fa0bc0a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:01.118261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.118921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-316f892d-c45d-4955-a177-fa131f72640d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.120725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1357/2702] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:00 2022] DELETE /identity/v3/projects/5f4cae215c6146ddaf1cab2fa0bc0a4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:01.125298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.125596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.125876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.144058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:01.145286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.204892 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1346/2703] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:01.213210 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.214053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.214969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.234250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:01.235453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.243944 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1358/2704] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:01.248578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.258675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a7822-b90f-444d-90a4-bf5e927bd131 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.263771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.264448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.264736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.265021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.265746 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:01.266087 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.267791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.282505 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1347/2705] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:59:01.287248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.297557 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115e09-0515-4050-9e54-7620c8e6da79 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.302824 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.303489 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.303777 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.304059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.305304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.307260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.336880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1359/2706] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:01.341027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.363543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325c4ad-b606-4a5b-aa67-c1a33e36b73a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.375119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.381932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.399239 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:01.442595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.443269 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.443547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.444017 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.444698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.446851 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.467976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1348/2707] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:01.468723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f43c27-4871-4ea5-9a81-34aee899b9c5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.471981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.474877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.477384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.478146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.478723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] PATH_INFO: `/v3/users/8e9c10838f2f42a1bf94679dfbfa7a69` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.480221 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9c10838f2f42a1bf94679dfbfa7a69)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.482156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bf690-74d7-404e-a507-33e70508b267 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.487221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.488977 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.490947 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.491450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.492770 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.493896 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.496627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.512077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1349/2708] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:01.513929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.527885 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545567-e97f-4e4b-bcfd-b88a579d04e0 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.534986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.535986 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.536659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.537056 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.538234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.540591 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.552687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9c10838f2f42a1bf94679dfbfa7a69'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.553264 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1350/2709] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:01.557854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.567018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9c10838f2f42a1bf94679dfbfa7a69'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.569411 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235cc0d-cd54-4ce4-902f-3799371a79d5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.571609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9c10838f2f42a1bf94679dfbfa7a69'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.572437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9c10838f2f42a1bf94679dfbfa7a69'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.576515 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.577520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.577943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.578316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] PATH_INFO: `/v3/projects/f46b6b7352124b3fb7406e48bdbd094c/users/4b9e60370949432a9067c61a88a7e33f/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.597919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-71e6784c-1126-4079-b436-5470c3eaae64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9c10838f2f42a1bf94679dfbfa7a69'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:01.606668 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:01.627857 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1360/2710] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:01 2022] DELETE /identity/v3/users/8e9c10838f2f42a1bf94679dfbfa7a69 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:01.637227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f46b6b7352124b3fb7406e48bdbd094c, user_id=4b9e60370949432a9067c61a88a7e33f, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.640380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.666626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1351/2711] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:01 2022] PUT /identity/v3/projects/f46b6b7352124b3fb7406e48bdbd094c/users/4b9e60370949432a9067c61a88a7e33f/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:01.670564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.670861 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.671145 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.704655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] MFA Rules not processed for user `4b9e60370949432a9067c61a88a7e33f`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:01.707027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.788763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1361/2712] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:01.792976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.794955 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.843306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:01.844713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:01.905630 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fb3bf-58c1-42a2-b017-63885d76c6b1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.913873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.914635 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.914919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.915197 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.915894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.918590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.942001 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3fd7e-3d12-4b4a-8fff-8b98350e8592 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:01.947457 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:01.948416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:01.948852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:01.949137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] PATH_INFO: `/v3/projects/0efd7c9275cf43b0b7c31325840a35b2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:01.950452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efd7c9275cf43b0b7c31325840a35b2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:01.952445 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:01.956163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1352/2713] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:01.957262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.011856 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efd7c9275cf43b0b7c31325840a35b2', 'request_id': 'req-19983506-22c9-450a-a3e9-1fed3c9eeaf4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:02.022695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efd7c9275cf43b0b7c31325840a35b2', 'request_id': 'req-19983506-22c9-450a-a3e9-1fed3c9eeaf4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:02.024413 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:02.028550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5760b0-e033-4487-8a16-25644143e20c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.035840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efd7c9275cf43b0b7c31325840a35b2', 'request_id': 'req-19983506-22c9-450a-a3e9-1fed3c9eeaf4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:02.051361 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] The token cache is being invalidate because project 0efd7c9275cf43b0b7c31325840a35b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:02.051954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:02.052843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-19983506-22c9-450a-a3e9-1fed3c9eeaf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:02.056708 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1362/2714] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:01 2022] DELETE /identity/v3/projects/0efd7c9275cf43b0b7c31325840a35b2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:02.062586 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf76d-1a96-43f8-9aba-01d8cb4359e2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.062968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf76d-1a96-43f8-9aba-01d8cb4359e2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.064281 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cf76d-1a96-43f8-9aba-01d8cb4359e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.077507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:02.093738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-aa9cf76d-1a96-43f8-9aba-01d8cb4359e2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:02.094795 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.095640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-aa9cf76d-1a96-43f8-9aba-01d8cb4359e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.097384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.097854 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.098873 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.100181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.102921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.107110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1363/2715] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:02.115051 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.115470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.115758 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.129790 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1353/2716] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:01 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:59:02.133929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.135579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.140247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:02.141760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.149848 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf3e2c-1ef0-48f6-88d8-0625b4ffed7c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.151380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1364/2717] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:02.155067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.155874 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.156173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.156491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.156917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.157818 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.159503 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.166622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4ac343-cf3a-4719-ac70-08ad06038865 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.171159 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.175050 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1354/2718] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.175216 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.176276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.187575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d273d-cb52-475a-81a6-a82836050a1e admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.192787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1365/2719] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:59:02.193676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.193801 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.194825 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.195206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.195570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] PATH_INFO: `/v3/projects/02fc39f752e848fd87838d5083c36e85/users/914769ce6105451790c508a01a7dd6c2/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.203887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710d3dc-e9fa-4749-954d-07239c049968 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.209485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.210622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.210918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.211199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.211804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fc39f752e848fd87838d5083c36e85, user_id=914769ce6105451790c508a01a7dd6c2, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.211937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.213790 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.214993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.243883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1355/2720] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:02 2022] PUT /identity/v3/projects/02fc39f752e848fd87838d5083c36e85/users/914769ce6105451790c508a01a7dd6c2/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:02.248063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.249262 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1366/2721] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:02.253312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.321265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86fec2-f242-458b-81d9-13f3bcd528e2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.328770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.329502 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.329789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.330141 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.330947 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.333075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.337237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca1fd5-0c9e-49a3-9fe9-e77256c57eb5 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.345744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.346644 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.347036 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.347878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.349510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.351407 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.361676 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1367/2722] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:02.362911 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.366700 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1356/2723] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:02.368642 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.374053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120902d-6b36-4596-bf33-fd9b2abb5b08 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.379538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.380645 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb7e42-588b-4b0b-9edf-9496874e77c4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.380782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.380782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.381009 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.383213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.385611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.387555 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.388251 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.388864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.389374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] PATH_INFO: `/v3/projects/02fc39f752e848fd87838d5083c36e85/users/914769ce6105451790c508a01a7dd6c2/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.405836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1368/2724] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:02.406883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.407766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fc39f752e848fd87838d5083c36e85, user_id=914769ce6105451790c508a01a7dd6c2, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.413511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd1ed9-3c3b-48ac-af3b-58ea58c4b35a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.442356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1357/2725] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:02 2022] PUT /identity/v3/projects/02fc39f752e848fd87838d5083c36e85/users/914769ce6105451790c508a01a7dd6c2/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:02.449203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48a27a-c2ff-4bde-984f-2d2da90f9db9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.467280 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.468265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.470744 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.541095 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.544454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.545250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.545458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.546475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.548332 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.571073 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] MFA Rules not processed for user `914769ce6105451790c508a01a7dd6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:02.571951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1369/2726] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:02.572459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.576964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.593254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167178-3472-4ebf-a1f7-9e8321c90ea8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.602945 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.604732 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.605156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.605639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] PATH_INFO: `/v3/projects/0e04ed1cada645e69e71fd50f44274ca/users/0ecf487c83584620ad1be47dc4b169a4/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.640373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e04ed1cada645e69e71fd50f44274ca, user_id=0ecf487c83584620ad1be47dc4b169a4, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.645424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94228512-f2c5-4b77-93a1-5ed8e8d31897 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.651987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:02.675555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1370/2727] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:02 2022] PUT /identity/v3/projects/0e04ed1cada645e69e71fd50f44274ca/users/0ecf487c83584620ad1be47dc4b169a4/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:02.684701 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.685235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.685865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.708593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:02.738403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1358/2728] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:02.741698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] MFA Rules not processed for user `0ecf487c83584620ad1be47dc4b169a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:02.743956 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.752869 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.754551 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.829032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1371/2729] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:02.840044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30164070-b148-4a14-b7a5-00575d7d680c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.841166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:02.842578 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:02.901155 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.902002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.902150 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.902460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.916854 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe24847-71dc-4e31-bd76-c65537a369f8 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:02.921428 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:02.922470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:02.922765 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:02.923080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:02.934710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.936179 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:02.950261 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1372/2730] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:02.985290 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:02.986962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:03.014108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1359/2731] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:03.130207 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:03.142901 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72415cbd-9592-4b41-8fa5-43c8ee8fe85d service glance] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:03.150939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:03.151687 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:03.152002 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:03.152312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:03.155120 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:03.229713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:03.236964 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d78a5-96f5-4288-8d85-736283261f1f service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:03.255137 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1373/2732] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:04.859720 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:04.861613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:04.900054 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:04.935188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450945f7-8185-40f0-b11d-d791295cb6fe service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:04.940677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:04.941615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:04.941906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:04.942350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] PATH_INFO: `/v3/users/4594073ff31d4bca9b11f5a13b1b9eea` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:04.943537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4594073ff31d4bca9b11f5a13b1b9eea)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:04.946164 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:04.998134 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4594073ff31d4bca9b11f5a13b1b9eea'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.010666 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4594073ff31d4bca9b11f5a13b1b9eea'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.011333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4594073ff31d4bca9b11f5a13b1b9eea'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.011936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4594073ff31d4bca9b11f5a13b1b9eea'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.019183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4594073ff31d4bca9b11f5a13b1b9eea'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.031977 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1360/2733] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:04 2022] DELETE /identity/v3/users/4594073ff31d4bca9b11f5a13b1b9eea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:05.036482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.036948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.037063 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.054706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:05.055940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.090535 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:05.117200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1374/2734] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:05.127421 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.128876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.186401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc7f98-5b6b-4e56-a4ed-75cd112ca3b9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:05.190090 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:05.190856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.191089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.191367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.201749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:05.203232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849513b-1aa8-47fd-b9da-fa5570aeed3e service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:05.213588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1361/2735] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:05.590229 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.591680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.613542 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595b4bd-bc21-4b6a-a6fa-a1439dfb569e None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:05.618583 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:05.619591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.619877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.620149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] PATH_INFO: `/v3/projects/ea6bafdd99b14f8abbee0d2f9f07b14b` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.621341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6bafdd99b14f8abbee0d2f9f07b14b)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:05.623334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:05.681206 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6bafdd99b14f8abbee0d2f9f07b14b', 'request_id': 'req-c33e417c-e3c1-4ef5-bc37-604dac88829c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.688738 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6bafdd99b14f8abbee0d2f9f07b14b', 'request_id': 'req-c33e417c-e3c1-4ef5-bc37-604dac88829c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.695799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6bafdd99b14f8abbee0d2f9f07b14b', 'request_id': 'req-c33e417c-e3c1-4ef5-bc37-604dac88829c'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.707003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] The token cache is being invalidate because project ea6bafdd99b14f8abbee0d2f9f07b14b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:05.707297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.707961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-c33e417c-e3c1-4ef5-bc37-604dac88829c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:05.709513 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1375/2736] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:05 2022] DELETE /identity/v3/projects/ea6bafdd99b14f8abbee0d2f9f07b14b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:05.714419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.714713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.714994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.732688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:05.733904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.783237 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:05.796756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1362/2737] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:05.805231 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.805538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.805815 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.824728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:05.825934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.833775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1376/2738] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:05.838001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.847815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8d0e5-960e-47f1-a9d0-db486ecd3ee3 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:05.853317 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:05.854031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.854323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.854603 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.855336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:05.855647 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:05.857272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:05.870223 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1363/2739] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:59:05.874575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.883794 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc2515-0dc7-457b-b02f-b7fb7777fdd5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:05.888377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:05.889020 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.889301 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.889575 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.890333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:05.892290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:05.919191 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1377/2740] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:05 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:05.923084 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.933937 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a730f-2c71-42fb-b6d8-8ed60c0524f7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:05.986953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:05.988767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:05.992656 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:05.993458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:05.993847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:05.994221 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:05.995005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:05.997058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.002306 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac6a0-b9ae-44fa-86ec-b264f80263b0 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.007046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.008437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.008743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.008996 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] PATH_INFO: `/v3/users/4b9e60370949432a9067c61a88a7e33f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.010618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9e60370949432a9067c61a88a7e33f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.012913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.018564 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1364/2741] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:05 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:06.022466 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.032763 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cc4d2-f135-4866-b99f-53d0a4ebada8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.037517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.038216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.038510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.038794 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.039530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.041262 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.051998 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1365/2742] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:06.056015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.066450 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9e60370949432a9067c61a88a7e33f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.066951 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018964bf-706b-4836-ba53-d144f1514a74 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.072045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.072691 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.072978 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.073356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.074107 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.074232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9e60370949432a9067c61a88a7e33f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.074951 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9e60370949432a9067c61a88a7e33f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.075647 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9e60370949432a9067c61a88a7e33f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.076265 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.085072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-a843d60e-3402-4224-ba5e-96d49d75b892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9e60370949432a9067c61a88a7e33f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.094312 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1366/2743] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:06.095626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.097341 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.104916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1378/2744] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:05 2022] DELETE /identity/v3/users/4b9e60370949432a9067c61a88a7e33f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:06.109085 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.109379 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.109661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.110438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef195c7-4ac9-4581-acdb-acd4b9f46c65 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.116054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:06.132087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:06.134558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.177749 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.178673 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.178954 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.179257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] PATH_INFO: `/v3/projects/c1f37e040d814faeb8c3b38bfcb372bc/users/73946412a02d4fe8a5407d1d33dcb40a/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.194571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1379/2745] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:06.195488 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f37e040d814faeb8c3b38bfcb372bc, user_id=73946412a02d4fe8a5407d1d33dcb40a, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.198294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7b5d7-4e15-4238-9e38-0e4028d6d835 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.204918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.206918 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.213688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1367/2746] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:06 2022] PUT /identity/v3/projects/c1f37e040d814faeb8c3b38bfcb372bc/users/73946412a02d4fe8a5407d1d33dcb40a/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:06.224756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.224972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.225247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.256373 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] MFA Rules not processed for user `73946412a02d4fe8a5407d1d33dcb40a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:06.258217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.271121 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb46b1-dc4a-47a4-8a4c-8f628ec20990 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.279762 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:06.326181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.326845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.327136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.327418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.327571 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1368/2747] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:06.341108 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.342536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.400062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.400961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db001347-addb-491f-966d-48e7ac1456f0 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.401526 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.404799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.405449 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.405736 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.406051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.411303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1380/2748] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:06.415417 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.416911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.426472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1369/2749] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:06.776176 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.778125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:06.797713 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0990c-563a-4ae3-8b0f-e1035a6a3768 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.804242 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.805789 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.806205 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.806563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] PATH_INFO: `/v3/projects/f46b6b7352124b3fb7406e48bdbd094c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.809852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f46b6b7352124b3fb7406e48bdbd094c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.812390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:06.829162 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=100 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:06.874435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f46b6b7352124b3fb7406e48bdbd094c', 'request_id': 'req-7c463676-9db1-48b4-ab4e-77cf5639e739'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.881694 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f46b6b7352124b3fb7406e48bdbd094c', 'request_id': 'req-7c463676-9db1-48b4-ab4e-77cf5639e739'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.887844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f46b6b7352124b3fb7406e48bdbd094c', 'request_id': 'req-7c463676-9db1-48b4-ab4e-77cf5639e739'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.898251 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] The token cache is being invalidate because project f46b6b7352124b3fb7406e48bdbd094c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:06.898531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.899153 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:06.900678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1381/2750] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:06 2022] DELETE /identity/v3/projects/f46b6b7352124b3fb7406e48bdbd094c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:06.904842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:06.916390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbfc63-c0c8-4c02-8070-ff470f5c0eb1 service cinder] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:06.972589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:06.973491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:06.973778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:06.974085 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] PATH_INFO: `/v3/users/914769ce6105451790c508a01a7dd6c2` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:06.975279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914769ce6105451790c508a01a7dd6c2)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:06.977836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.023876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914769ce6105451790c508a01a7dd6c2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.035273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914769ce6105451790c508a01a7dd6c2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.035918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914769ce6105451790c508a01a7dd6c2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.036517 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914769ce6105451790c508a01a7dd6c2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.039072 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:07.044817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-be231749-62c4-4ba1-afef-c8890a7ee7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914769ce6105451790c508a01a7dd6c2'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.059174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1370/2751] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:06 2022] DELETE /identity/v3/users/914769ce6105451790c508a01a7dd6c2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:07.337443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.338940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:07.358504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463676-9db1-48b4-ab4e-77cf5639e739 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.419420 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.420283 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.420564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.420840 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] PATH_INFO: `/v3/projects/02fc39f752e848fd87838d5083c36e85` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.422040 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fc39f752e848fd87838d5083c36e85)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.423960 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.471093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fc39f752e848fd87838d5083c36e85', 'request_id': 'req-121a8bca-520e-40b8-b81f-55f8c9b2e2be'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.478117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fc39f752e848fd87838d5083c36e85', 'request_id': 'req-121a8bca-520e-40b8-b81f-55f8c9b2e2be'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.484165 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fc39f752e848fd87838d5083c36e85', 'request_id': 'req-121a8bca-520e-40b8-b81f-55f8c9b2e2be'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.496327 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] The token cache is being invalidate because project 02fc39f752e848fd87838d5083c36e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:07.496669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.497717 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-121a8bca-520e-40b8-b81f-55f8c9b2e2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:07.499715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1382/2752] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:07 2022] DELETE /identity/v3/projects/02fc39f752e848fd87838d5083c36e85 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:07.504952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.505313 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.505622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.524181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:07.525385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:07.586587 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1371/2753] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:07.594685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.594971 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.595245 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.614486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:07.615697 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:07.624172 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1383/2754] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:07.628559 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.638716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be215252-94de-4120-b7a0-38be342b85e9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.643386 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.644039 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.644323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.644599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.645321 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:07.645627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.647548 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.659266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1372/2755] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:59:07.664490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.674480 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cbd1e-fd3b-4287-864d-d3b0ba687e86 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.680014 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.680521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.680839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.681117 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.681860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.683823 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.698273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:07.714836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1384/2756] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:07 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:07.718856 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.730570 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cb05b-6b3f-4c98-b8dd-9736f6a3b9a0 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.799860 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.800653 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.801040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.801324 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.802222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.804581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.827500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1373/2757] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:07 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:59:07.832090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.845493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31f25-612b-4ffb-9c68-06b379bf69a1 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.853530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.854392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.854706 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.855083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.855750 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.857462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.869972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1385/2758] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:07.873441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.884473 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d1bf0-2d84-485c-b36e-23103bc93bee admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.889433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.890198 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.890549 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.890895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.891692 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.893371 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8c654-9a2a-4a53-8f28-53874624205e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.904167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1374/2759] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:07.908521 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:07.918928 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65938e7-cf9c-415c-945d-57cf9944d292 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:07.923570 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:07.924556 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.924836 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.925108 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] PATH_INFO: `/v3/projects/dd3c2c3343c04857a59ed8ee228287df/users/6ddde3d5d23041dbab7219cd8c928e13/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.940609 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd3c2c3343c04857a59ed8ee228287df, user_id=6ddde3d5d23041dbab7219cd8c928e13, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:07.943986 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:07.957780 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1386/2760] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:07 2022] PUT /identity/v3/projects/dd3c2c3343c04857a59ed8ee228287df/users/6ddde3d5d23041dbab7219cd8c928e13/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:07.967894 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:07.968403 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:07.968507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:07.997658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] MFA Rules not processed for user `6ddde3d5d23041dbab7219cd8c928e13`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:07.998921 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:08.043950 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:08.075234 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1375/2761] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:08.079999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.150235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef38c-9e19-476b-b713-cd7069ab6a81 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.155077 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.155734 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.156013 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.156292 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.156990 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.158934 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.187119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1387/2762] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:08 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:59:08.191005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.202040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262fccd-f47a-4743-9afe-3b02d7ca9cf9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.270705 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.271650 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.271945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.272230 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.273595 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.275519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.295502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1376/2763] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:08 2022] POST /identity/v3/users => generated 301 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:08.299814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.310538 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf144-283b-4718-85b6-988d0c9b5dd9 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.315876 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.316582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.317081 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.317413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.318367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.321273 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.338670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1388/2764] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:08.339655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.353626 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c03e4d-fdd9-479b-a068-239c29c4928a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.355902 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.356905 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.357295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.357712 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] PATH_INFO: `/v3/projects/2f0572966b0f4525acea8351ac375b54/users/c9f22fd0395a4c4083c9b546af5fb4a4/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.375088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0572966b0f4525acea8351ac375b54, user_id=c9f22fd0395a4c4083c9b546af5fb4a4, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.377772 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.402000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1377/2765] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:08 2022] PUT /identity/v3/projects/2f0572966b0f4525acea8351ac375b54/users/c9f22fd0395a4c4083c9b546af5fb4a4/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:08.404424 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.415296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7dddd-1ad0-4659-99dc-a9da90560169 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.472091 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.472759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.473048 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.473326 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.474082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.475684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782e527-3f55-4fa2-8454-cb510bfd7ae8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.486086 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1389/2766] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:08.490215 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.500277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c6d25-0a53-4451-8113-ba0519fba41c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.504871 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.505730 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.506038 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.506323 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] PATH_INFO: `/v3/projects/2f0572966b0f4525acea8351ac375b54/users/c9f22fd0395a4c4083c9b546af5fb4a4/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.508291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0572966b0f4525acea8351ac375b54, user_id=c9f22fd0395a4c4083c9b546af5fb4a4, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.510982 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.521538 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1378/2767] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:08 2022] PUT /identity/v3/projects/2f0572966b0f4525acea8351ac375b54/users/c9f22fd0395a4c4083c9b546af5fb4a4/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:08.533565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.533859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.534284 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.560466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] MFA Rules not processed for user `c9f22fd0395a4c4083c9b546af5fb4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:08.561666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:08.579315 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:08.629897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1390/2768] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:08.642345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:08.644104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:08.702798 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60822b26-9632-46c8-9bdb-dca2152d82f7 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:08.706464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:08.707114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:08.707401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:08.707682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:08.717179 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:08.718689 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e0db9-b38d-407d-bbbc-b3fb17e62823 service cinder] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:08.728809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1379/2769] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:09.049921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:09.116097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914bcb7-d9df-4ce4-b6c4-1ad48504b455 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:09.120616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:09.121483 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:09.121763 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:09.122131 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] PATH_INFO: `/v3/users/6ddde3d5d23041dbab7219cd8c928e13` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:09.123313 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ddde3d5d23041dbab7219cd8c928e13)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:09.125240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:09.171859 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ddde3d5d23041dbab7219cd8c928e13'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.180080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ddde3d5d23041dbab7219cd8c928e13'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.180855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ddde3d5d23041dbab7219cd8c928e13'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.181463 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ddde3d5d23041dbab7219cd8c928e13'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.189360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ddde3d5d23041dbab7219cd8c928e13'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.207195 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1391/2770] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:09 2022] DELETE /identity/v3/users/6ddde3d5d23041dbab7219cd8c928e13 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:09.212207 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:09.212688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:09.212817 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:09.231423 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:09.232665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:09.263804 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:09.294743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1380/2771] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:09.304766 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:09.306211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:09.346979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:09.365880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d2740-ae09-44e5-9d81-16141a7e680d admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:09.370491 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:09.371202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:09.371614 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:09.371955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:09.382244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:09.384246 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:09.394046 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1392/2772] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:09.688836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:09.690400 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:09.704667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff04e-61d0-442d-a044-bd1f0894c3e9 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:09.709990 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:09.711448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:09.711716 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:09.711981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] PATH_INFO: `/v3/projects/dd3c2c3343c04857a59ed8ee228287df` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:09.713157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd3c2c3343c04857a59ed8ee228287df)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:09.715101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:09.768005 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd3c2c3343c04857a59ed8ee228287df', 'request_id': 'req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.774789 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd3c2c3343c04857a59ed8ee228287df', 'request_id': 'req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.781025 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd3c2c3343c04857a59ed8ee228287df', 'request_id': 'req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.792884 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] The token cache is being invalidate because project dd3c2c3343c04857a59ed8ee228287df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:09.793192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.793792 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.795376 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1381/2773] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:09 2022] DELETE /identity/v3/projects/dd3c2c3343c04857a59ed8ee228287df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:09.799571 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:09.809913 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac929813-eb50-4cdf-8fed-6e383ebe7fd2 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:09.870767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:09.871648 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:09.871935 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:09.872259 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] PATH_INFO: `/v3/users/c9f22fd0395a4c4083c9b546af5fb4a4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:09.873422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f22fd0395a4c4083c9b546af5fb4a4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:09.875407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:09.923728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f22fd0395a4c4083c9b546af5fb4a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.931244 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f22fd0395a4c4083c9b546af5fb4a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.931968 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f22fd0395a4c4083c9b546af5fb4a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.932625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f22fd0395a4c4083c9b546af5fb4a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.940434 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-d6c05dce-9a57-4461-93de-ecefdc61add6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f22fd0395a4c4083c9b546af5fb4a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:09.958437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1393/2774] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:09 2022] DELETE /identity/v3/users/c9f22fd0395a4c4083c9b546af5fb4a4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:10.252699 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.253665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:10.267936 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a4c0a-37a4-4625-a66e-1a180d7c7e1c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.327847 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.328719 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.329008 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.329287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] PATH_INFO: `/v3/projects/2f0572966b0f4525acea8351ac375b54` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.330520 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0572966b0f4525acea8351ac375b54)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.332578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.380295 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0572966b0f4525acea8351ac375b54', 'request_id': 'req-eac02e36-ed32-4453-8eb5-d08f57e4defa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:10.386781 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0572966b0f4525acea8351ac375b54', 'request_id': 'req-eac02e36-ed32-4453-8eb5-d08f57e4defa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:10.391823 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:10.393363 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0572966b0f4525acea8351ac375b54', 'request_id': 'req-eac02e36-ed32-4453-8eb5-d08f57e4defa'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:10.408367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] The token cache is being invalidate because project 2f0572966b0f4525acea8351ac375b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:10.408661 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:10.409287 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-eac02e36-ed32-4453-8eb5-d08f57e4defa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:10.410864 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1382/2775] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:10 2022] DELETE /identity/v3/projects/2f0572966b0f4525acea8351ac375b54 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:10.416529 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.417024 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.417132 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.434465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:10.435663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:10.487082 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:10.498474 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1394/2776] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:10.506592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.506906 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.507713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.527153 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:10.528388 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:10.536366 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1383/2777] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:10.540562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.550558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced2676-d0e5-466f-898c-7a6254fee69a None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.555260 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.555927 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.556224 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.556506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.557239 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:10.557555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.559200 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.571803 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1395/2778] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:59:10.576257 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.587380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26e167-9a6e-46f5-81b1-0d48911cb4ba None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.592297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.592967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.593271 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.593550 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.594284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.596220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.624611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1384/2779] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:10 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:10.628841 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.640184 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd9419-d6d0-4c65-90e2-0aaea437cf4b admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.701744 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.702568 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.702860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.703144 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.704053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.707460 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.733994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1396/2780] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:10 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:10.735547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.747766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4c1b-9bd8-46b4-b12f-5f31bf098231 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.752354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.753097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.753390 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.753665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.754425 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.756077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.767625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1385/2781] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:10.772299 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.782656 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b456ce-8aee-46b9-acfa-63fc29154894 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.787472 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.788134 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.788443 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.788726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.789640 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.791340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffacc4e-fcf0-4e45-a321-14aab066029e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.802311 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1397/2782] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:10.807055 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.817129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e2834-93bf-428c-80ac-41cd27abc2db admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:10.822487 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:10.823356 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.823649 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.823931 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] PATH_INFO: `/v3/projects/e4c3d69670114e9dae7efa806fa631b8/users/bb6079e422754567a1cbcd3d806ec1c3/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.839176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c3d69670114e9dae7efa806fa631b8, user_id=bb6079e422754567a1cbcd3d806ec1c3, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:10.842001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:10.855934 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1386/2783] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:10 2022] PUT /identity/v3/projects/e4c3d69670114e9dae7efa806fa631b8/users/bb6079e422754567a1cbcd3d806ec1c3/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:10.865623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:10.865916 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:10.866282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:10.892371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] MFA Rules not processed for user `bb6079e422754567a1cbcd3d806ec1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:10.893611 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:10.969651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1398/2784] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:10.980667 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:10.982446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:11.041840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1d27-edd7-4b13-b9dc-d72e0fb69038 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:11.045552 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:11.046227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:11.046527 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:11.046808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:11.055733 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:11.057172 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:11.066024 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1387/2785] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:18.785147 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:18.786576 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:18.843339 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee57ae-e40d-46fe-b7cf-08feb826cc14 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:18.847366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:18.848034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:18.848317 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:18.848593 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:18.850955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:18.857845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:18.922672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:18.924107 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:18.933930 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1399/2786] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:20.568803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:20.570734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:20.648919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e3d0f6-94ec-4e70-8092-5d21132e504b service glance] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:20.654461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:20.655109 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:20.655401 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:20.655680 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:20.657446 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:20.671320 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:20.672927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf2ce4-351a-46e4-a3b3-b712adbadd4b service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:20.685048 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1388/2787] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:28.929721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=26 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:30.679803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:35.070837 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.072258 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.136605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9479e-b449-4ba3-901e-7df7ce33c955 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:35.141709 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:35.142621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.142929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.143203 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] PATH_INFO: `/v3/users/0ecf487c83584620ad1be47dc4b169a4` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.144459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecf487c83584620ad1be47dc4b169a4)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:35.147036 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:35.194800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecf487c83584620ad1be47dc4b169a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.202623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecf487c83584620ad1be47dc4b169a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.203310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecf487c83584620ad1be47dc4b169a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.203937 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecf487c83584620ad1be47dc4b169a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.211213 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecf487c83584620ad1be47dc4b169a4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.230057 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1400/2788] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:35 2022] DELETE /identity/v3/users/0ecf487c83584620ad1be47dc4b169a4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:35.234787 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.235286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.235512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.253274 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:35.254601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.315615 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1389/2789] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:35.326523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.327961 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.385966 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8005e-0f4f-4e6d-8b2c-f069cde22457 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:35.389711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:35.390462 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.390747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.391027 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.401413 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:35.402962 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4fa15-43cf-4bbc-8437-b8880c0110d5 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:35.413044 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1401/2790] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:35.683979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.685416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.699747 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492275f-1f7d-4ec1-9f55-3bfca469728b None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:35.711114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:35.712031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.712222 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.712496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] PATH_INFO: `/v3/projects/0e04ed1cada645e69e71fd50f44274ca` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.713726 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e04ed1cada645e69e71fd50f44274ca)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:35.715690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:35.761862 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e04ed1cada645e69e71fd50f44274ca', 'request_id': 'req-a7eeaacf-8293-4534-8535-009cbd6e1f69'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.768000 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e04ed1cada645e69e71fd50f44274ca', 'request_id': 'req-a7eeaacf-8293-4534-8535-009cbd6e1f69'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.773753 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e04ed1cada645e69e71fd50f44274ca', 'request_id': 'req-a7eeaacf-8293-4534-8535-009cbd6e1f69'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.788393 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] The token cache is being invalidate because project 0e04ed1cada645e69e71fd50f44274ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:35.788683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.789304 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-a7eeaacf-8293-4534-8535-009cbd6e1f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:35.790889 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1390/2791] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:35 2022] DELETE /identity/v3/projects/0e04ed1cada645e69e71fd50f44274ca => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:35.796371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.796661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.796939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.815397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:35.816598 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.834032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:35.880297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1402/2792] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:35.889756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.890083 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.890362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.908880 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:35.910142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.918584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1391/2793] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:35.922787 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.932664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dfdd4-e3be-46dd-80bc-1df02915e345 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:35.937771 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:35.938451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.938737 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.939019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.939743 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91697) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:35.940052 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:35.941655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:35.953623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1403/2794] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 10:59:35.957713 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.967690 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287290a7-4deb-47be-ae9b-4cbd9f624d74 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:35.975015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:35.975678 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:35.975962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:35.976241 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:35.976973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:35.978939 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:35.983388 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:35.985122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:35.998830 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e476-76cf-464c-940a-4006dfcbcc81 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.003530 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.004422 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.004710 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.004992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] PATH_INFO: `/v3/users/bb6079e422754567a1cbcd3d806ec1c3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.006240 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6079e422754567a1cbcd3d806ec1c3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.008187 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.012930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1392/2795] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:35 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:36.014596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.057976 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6079e422754567a1cbcd3d806ec1c3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.066399 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6079e422754567a1cbcd3d806ec1c3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.067243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6079e422754567a1cbcd3d806ec1c3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.067865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6079e422754567a1cbcd3d806ec1c3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.074354 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:36.077366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-6ab845f4-0082-475e-9589-bad1439ddd99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6079e422754567a1cbcd3d806ec1c3'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.081086 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abd6b5-6d35-424c-9aba-06699d26ac18 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.087926 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.088542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.088827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.089106 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.089796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.091796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.097783 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1404/2796] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:35 2022] DELETE /identity/v3/users/bb6079e422754567a1cbcd3d806ec1c3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:36.102089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.102376 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.102650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.114916 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1393/2797] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 10:59:36.118668 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.120374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.120849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:36.122076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.132785 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33693505-d9c8-4644-804d-ae149c86cbd9 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.188133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1405/2798] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 10:59:36.191995 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.192670 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.192961 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.193244 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.193993 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.195619 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.197805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.199175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.209191 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1394/2799] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:36.210523 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.222963 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522799b-12a8-41c6-92af-e3c064d8c5db admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.227797 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.228460 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.228750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.229031 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.229758 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.230597 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:36.235930 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.242052 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1395/2800] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:36.245967 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.256069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ce3e-9b2c-4ae4-92cf-328c784c34d3 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.261297 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.262584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.263007 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.266002 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] PATH_INFO: `/v3/projects/0c7e29a1e59b43a4a35f700a7e1be42c/users/dd2c5c03daed4679afee0143e6e99a68/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.266163 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5caf3-89ba-4bfa-9407-460dd43dd1e4 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.270125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.270822 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.271372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.271651 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.281639 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7e29a1e59b43a4a35f700a7e1be42c, user_id=dd2c5c03daed4679afee0143e6e99a68, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.282122 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.283560 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146931d-808b-453a-9abc-d68a6dc8587e service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.284381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.293622 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1406/2801] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:36.303097 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1396/2802] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:36 2022] PUT /identity/v3/projects/0c7e29a1e59b43a4a35f700a7e1be42c/users/dd2c5c03daed4679afee0143e6e99a68/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:36.307919 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.308214 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.308495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.326097 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:36.328186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.401454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1407/2803] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:36.410420 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.412227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.474694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee577a-7b85-43a5-9641-4d7b153da055 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.478671 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.479334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.479622 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.479900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.490384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.491828 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410c9bd-fd99-40a2-8a6b-6cb636decc3d service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.501583 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1397/2804] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 10:59:36.621849 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.623509 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.637421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a266518-12b9-4308-b1c7-d1503b1225d5 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.643415 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.644582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.644942 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.645512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] PATH_INFO: `/v3/projects/e4c3d69670114e9dae7efa806fa631b8` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.647256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c3d69670114e9dae7efa806fa631b8)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.649479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.709106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c3d69670114e9dae7efa806fa631b8', 'request_id': 'req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.717716 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c3d69670114e9dae7efa806fa631b8', 'request_id': 'req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.731558 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c3d69670114e9dae7efa806fa631b8', 'request_id': 'req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.743350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] The token cache is being invalidate because project e4c3d69670114e9dae7efa806fa631b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:36.743739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.744490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-4e5700c2-71dd-4bd9-a4ee-0ea009baf729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:36.746124 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1408/2805] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:36 2022] DELETE /identity/v3/projects/e4c3d69670114e9dae7efa806fa631b8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:36.753478 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.753771 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.754088 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.773803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:36.775102 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.826677 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:36.844391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1398/2806] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:36.852340 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.852621 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.852899 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.872941 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:36.874888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:36.883021 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1409/2807] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:36.887512 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.898064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942db26-77da-4bb7-87f4-fd85d7d92a15 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.902588 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.903255 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.903535 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.903822 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.904533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 10:59:36.904838 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.906508 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.922510 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1399/2808] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 10:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 10:59:36.923475 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.933059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818571c-d01e-4dc5-9897-c29f1c8120e2 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:36.938449 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:36.939110 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:36.939390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:36.939661 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:36.940454 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:36.942414 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:36.975444 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1410/2809] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:36.979592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:36.990723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ff340-5bf1-4849-b119-4e7d78d682df admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.057148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.057859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.058292 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.058492 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.059327 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.061569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.085249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1400/2810] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:36 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:37.089506 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:37.102650 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d3919-4a8a-4a44-b6ba-eae15c3defc8 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.107406 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.108164 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.108520 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.108865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.109658 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.111356 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.122232 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1411/2811] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 10:59:37.126142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:37.140151 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d96f-cbf5-4505-9fda-548a3e96e85d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.145643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.146702 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.147064 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.147434 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.148338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.150040 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b035f9d-85e7-4f3e-aea7-3ed5f2d14fc8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.162249 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1401/2812] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:37.166806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:37.177825 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43b566-632d-496d-8d22-4c569a89e8b0 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.182877 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.183847 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.184198 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.184580 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] PATH_INFO: `/v3/projects/86e332b470c841db8d7d852c1c0a4a26/users/5b53f05946e34f9f8395526212c5b17f/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.189357 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:37.209249 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e332b470c841db8d7d852c1c0a4a26, user_id=5b53f05946e34f9f8395526212c5b17f, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.212695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.229432 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1412/2813] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:37 2022] PUT /identity/v3/projects/86e332b470c841db8d7d852c1c0a4a26/users/5b53f05946e34f9f8395526212c5b17f/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:37.234578 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173da399-a294-4ef9-a4b8-e5aff6dfa4ca None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.234881 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173da399-a294-4ef9-a4b8-e5aff6dfa4ca None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.235475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173da399-a294-4ef9-a4b8-e5aff6dfa4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.259536 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-173da399-a294-4ef9-a4b8-e5aff6dfa4ca None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:37.260768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-173da399-a294-4ef9-a4b8-e5aff6dfa4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:37.325528 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1402/2814] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:37.335793 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:37.337688 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:37.399193 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6d52b-533d-49fb-a9f0-c72d6a94da7a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.403561 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.403875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.404178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.405089 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.415202 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.416722 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.428705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1413/2815] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:37.507061 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0318d2-9f25-4921-9595-b04e2cccf455 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.508454 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0318d2-9f25-4921-9595-b04e2cccf455 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.508751 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0318d2-9f25-4921-9595-b04e2cccf455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.541584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-9b0318d2-9f25-4921-9595-b04e2cccf455 None None] MFA Rules not processed for user `dd2c5c03daed4679afee0143e6e99a68`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:37.542849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-9b0318d2-9f25-4921-9595-b04e2cccf455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:37.601842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:37.631657 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1403/2816] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:37.642969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:37.644397 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:37.745866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923912f-bf06-4bda-8bb1-dbce362210d0 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:37.750226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4b93133aa146bca7ee0f205cf623a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:37.751039 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:37.751421 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:37.751769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:37.761458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:37.762973 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec6e2a-8efe-4ff9-b078-05d18b3668c0 service cinder] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:37.772456 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1414/2817] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:39.208755 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:39.209315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:39.209315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:39.228458 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] MFA Rules not processed for user `dd2c5c03daed4679afee0143e6e99a68`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:39.229686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:39.253243 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1404/2818] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:41.882805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:41.883269 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:41.883582 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:41.911747 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] MFA Rules not processed for user `5b53f05946e34f9f8395526212c5b17f`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:41.913041 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:41.984436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1415/2819] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:41.988895 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.001217 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1686e0-b5cd-46e4-9803-405be9334cd5 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.009202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.010142 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.010658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.011006 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.011840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.014913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.058275 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1405/2820] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 10:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:42.062925 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.076325 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afbd74-425e-4263-8c87-d69ad1162b91 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.100003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:42.149814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.150842 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.151287 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.151778 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] PATH_INFO: `/v3/users` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.152485 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.154703 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.179563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1416/2821] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 10:59:42 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 10:59:42.183596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.195159 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48db2-4431-45c5-9b67-9d3952479e79 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.200475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.201199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.201486 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.201766 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.202530 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.204188 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.216339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1406/2822] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:42.220891 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.240390 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5c2bc-7840-4b01-850a-970605edf660 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.249075 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.250392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.250392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.250605 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] PATH_INFO: `/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8/users/d23769a34b2f420db1613c6b270b9117/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.275360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2d9e0aa87a4791b8088a598f3e84b8, user_id=d23769a34b2f420db1613c6b270b9117, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.281431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.310042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1417/2823] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:42 2022] PUT /identity/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8/users/d23769a34b2f420db1613c6b270b9117/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:42.313659 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.325233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc279ad-1c0c-4a13-9b89-974e517b5acb admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.430715 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.431893 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.432381 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.432839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.433932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.436310 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.460545 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1407/2824] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 10:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 10:59:42.461296 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:42.473248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec94f19-d274-41d1-b32f-dc0f1839b417 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:42.480799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:42.481728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:42.482066 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:42.482442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] PATH_INFO: `/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8/users/d23769a34b2f420db1613c6b270b9117/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:42.484548 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e2d9e0aa87a4791b8088a598f3e84b8, user_id=d23769a34b2f420db1613c6b270b9117, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:42.487466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209005d-3987-4fcc-9f30-f6254c71c846 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:42.500062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1418/2825] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 10:59:42 2022] PUT /identity/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8/users/d23769a34b2f420db1613c6b270b9117/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 10:59:46.653697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:46.656960 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:46.825919 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851491c-ac69-44a4-af0c-8f5774398bcf admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:46.908338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:46.951533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:46.952957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:46.954416 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:46.954710 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:46.961675 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:46.983875 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:46.986447 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:47.003648 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1408/2826] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:47.989975 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:47.990470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:47.990470 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:48.019189 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] MFA Rules not processed for user `d23769a34b2f420db1613c6b270b9117`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:48.020437 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:48.094632 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1419/2827] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:48.109983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:48.111444 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:48.192735 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769b8c8-f16f-42c8-98b9-a981ed3e94a6 service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:48.198624 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:48.199291 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:48.199575 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:48.199852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:48.268247 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:48.269878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6f81a-fec8-4e80-bef3-f788df0232ea service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:48.279115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1409/2828] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:48.774795 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:48.777431 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:48.897341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:48.905093 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a401e-0891-4c5e-82fc-83f38eaf84ca None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:48.921113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:48.922138 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:48.922639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:48.922984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] PATH_INFO: `/v3/users/73946412a02d4fe8a5407d1d33dcb40a` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:48.924559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73946412a02d4fe8a5407d1d33dcb40a)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:48.926669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:49.015383 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73946412a02d4fe8a5407d1d33dcb40a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.025433 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73946412a02d4fe8a5407d1d33dcb40a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.026639 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73946412a02d4fe8a5407d1d33dcb40a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.027495 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73946412a02d4fe8a5407d1d33dcb40a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.041334 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73946412a02d4fe8a5407d1d33dcb40a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.094589 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1420/2829] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 10:59:48 2022] DELETE /identity/v3/users/73946412a02d4fe8a5407d1d33dcb40a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:49.099286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:49.099768 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:49.100199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:49.140769 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 10:59:49.143364 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:49.266414 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1410/2830] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 10:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 10:59:49.285819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:49.287525 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:49.379303 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437d2d-ba21-4392-8c9e-c00747bfefe4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:49.384585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:49.385395 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:49.386038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:49.386038 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:49.403972 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:49.405673 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2317b2-cb7d-4d51-82d1-4c8c82268ce5 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:49.420050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1421/2831] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 10:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 10:59:49.840284 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 10:59:49.842694 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 10:59:49.855059 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b3c5d-3314-4f1f-a0b9-e8231429083f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 10:59:49.860286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 10:59:49.861286 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 10:59:49.861637 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 10:59:49.862001 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] PATH_INFO: `/v3/projects/c1f37e040d814faeb8c3b38bfcb372bc` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 10:59:49.863469 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f37e040d814faeb8c3b38bfcb372bc)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 10:59:49.865496 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 10:59:49.873600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:49.920697 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f37e040d814faeb8c3b38bfcb372bc', 'request_id': 'req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.929614 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f37e040d814faeb8c3b38bfcb372bc', 'request_id': 'req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.936537 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f37e040d814faeb8c3b38bfcb372bc', 'request_id': 'req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.954553 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] The token cache is being invalidate because project c1f37e040d814faeb8c3b38bfcb372bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 10:59:49.955394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.955778 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 10:59:49.957732 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1411/2832] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 10:59:49 2022] DELETE /identity/v3/projects/c1f37e040d814faeb8c3b38bfcb372bc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 10:59:59.414072 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 10:59:59.949935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:00:10.229785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475594b7-52ef-42b6-a88e-05cb06175f8d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:00:10.230338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475594b7-52ef-42b6-a88e-05cb06175f8d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:00:10.230461 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475594b7-52ef-42b6-a88e-05cb06175f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:00:10.248377 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-475594b7-52ef-42b6-a88e-05cb06175f8d None None] MFA Rules not processed for user `5b53f05946e34f9f8395526212c5b17f`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:00:10.249564 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-475594b7-52ef-42b6-a88e-05cb06175f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:00:10.303034 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1422/2833] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:00:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:00:20.296551 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=25 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:00:32.191117 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:00:32.192519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:00:32.249803 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687f57-911c-4fa9-81ce-fb3f5393c2f4 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:00:32.253459 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:00:32.254133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:00:32.254419 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:00:32.254696 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:00:32.256484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:00:32.318543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:00:32.319976 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:00:32.329314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1412/2834] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:00:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:00:42.325114 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=50 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:00:50.865186 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:00:50.865726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:00:50.865726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:00:50.883591 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] MFA Rules not processed for user `dd2c5c03daed4679afee0143e6e99a68`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:00:50.884800 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:00:50.939282 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1423/2835] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:00:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:00.932590 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=25 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:40.461748 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:40.470472 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fd109-ed57-4853-95a6-9ec10a6e6413 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:40.474983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:40.475632 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:40.475920 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:40.476202 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:40.478063 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:40.547809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:40.549248 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:40.560078 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1413/2836] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:41.731166 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:41.732685 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:41.745372 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a1df4-b4ff-4d77-9044-a39e392c113f None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:41.748921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:41.749563 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:41.749868 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:41.750177 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:41.751932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:41.765007 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:41.766479 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:41.777606 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1424/2837] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:48.189662 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:48.191173 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:48.256827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c772320-a0f4-46ad-812d-4fe7bf1dcb61 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:48.261362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:48.262277 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.262561 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.262836 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] PATH_INFO: `/v3/users/dd2c5c03daed4679afee0143e6e99a68` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.264012 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2c5c03daed4679afee0143e6e99a68)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:48.265935 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:48.310470 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2c5c03daed4679afee0143e6e99a68'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.322456 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2c5c03daed4679afee0143e6e99a68'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.323192 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2c5c03daed4679afee0143e6e99a68'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.323833 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2c5c03daed4679afee0143e6e99a68'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.330979 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-722afc1e-1331-4692-8eb2-20d1fb91886f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2c5c03daed4679afee0143e6e99a68'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.343633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1414/2838] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:01:48 2022] DELETE /identity/v3/users/dd2c5c03daed4679afee0143e6e99a68 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:01:48.517350 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:48.528931 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8cdb7-d941-409b-9c4b-1d1415412f6a service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:48.586982 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:48.587863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.588154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.588436 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] PATH_INFO: `/v3/projects/0c7e29a1e59b43a4a35f700a7e1be42c` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.589655 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7e29a1e59b43a4a35f700a7e1be42c)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:48.591659 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:48.641459 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7e29a1e59b43a4a35f700a7e1be42c', 'request_id': 'req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.649369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7e29a1e59b43a4a35f700a7e1be42c', 'request_id': 'req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.655912 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7e29a1e59b43a4a35f700a7e1be42c', 'request_id': 'req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.666799 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] The token cache is being invalidate because project 0c7e29a1e59b43a4a35f700a7e1be42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:01:48.667084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.668101 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-aaa54f60-1dff-4ab0-9fd2-3ff3fb0fa87a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:01:48.669943 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1425/2839] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:01:48 2022] DELETE /identity/v3/projects/0c7e29a1e59b43a4a35f700a7e1be42c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:01:48.674611 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.674911 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.675266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.683384 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:48.696596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:48.698507 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:48.758101 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1415/2840] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:48.766069 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.766371 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.766721 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.785453 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:48.786705 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:48.794839 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1426/2841] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:48.799026 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:48.809451 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7584b-afee-4946-a6a5-02e2a1e2f41a None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:48.814156 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:48.814811 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.815100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.815406 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.816133 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91696) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 11:01:48.816442 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:48.818743 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:48.830970 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1416/2842] 10.222.0.36 () {58 vars in 1152 bytes} [Mon May 23 11:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 11:01:48.835427 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:48.844811 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f94b2b-08b7-47f9-a641-5ee8eebe3c4c None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:48.849775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:48.850559 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.850915 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.851270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.851984 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:48.853910 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:48.884183 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1427/2843] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 11:01:48 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 11:01:48.888220 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:48.899698 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b5ad1-b8b2-4041-b503-0e8751145ed8 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:48.958722 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:48.959519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:48.959815 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:48.960100 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:48.960837 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:48.962829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:48.985044 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1417/2844] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 11:01:48 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 11:01:48.989451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:49.000490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9a9b3-b443-4a6b-aa7b-ca9617801257 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:49.005263 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:49.005929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:49.006256 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:49.006537 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:49.007265 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:49.008873 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:49.019373 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1428/2845] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 11:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 11:01:49.023547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:49.033686 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8902318-dc6f-4516-808f-4053d7ee4a3c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:49.039160 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:49.039830 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:49.040115 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:49.040392 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:49.041250 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:49.042953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcb610-8820-4499-a6fc-5fa7c6d37d9b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:49.058016 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1418/2846] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 11:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 11:01:49.060504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:49.070760 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642524b-ff40-4434-847a-98d9c4d98b91 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:49.075845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:49.076731 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:49.077018 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:49.077423 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] PATH_INFO: `/v3/projects/e35c886cdaaf465eacd001e132128cbd/users/88701ef1915d4ad2955a4bf31fe7634b/roles/63df59ac99514b3983611a6b633ec83f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:49.092806 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35c886cdaaf465eacd001e132128cbd, user_id=88701ef1915d4ad2955a4bf31fe7634b, role_id=63df59ac99514b3983611a6b633ec83f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:49.095565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:49.109887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1429/2847] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 11:01:49 2022] PUT /identity/v3/projects/e35c886cdaaf465eacd001e132128cbd/users/88701ef1915d4ad2955a4bf31fe7634b/roles/63df59ac99514b3983611a6b633ec83f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 11:01:49.115604 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eab228-8140-4950-b5d1-8e78b0fbdb4c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:49.116103 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eab228-8140-4950-b5d1-8e78b0fbdb4c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:49.116461 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eab228-8140-4950-b5d1-8e78b0fbdb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:49.133491 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-21eab228-8140-4950-b5d1-8e78b0fbdb4c None None] MFA Rules not processed for user `eb9753922dc94eb786d2155e45d2454a`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:49.134849 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-21eab228-8140-4950-b5d1-8e78b0fbdb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:49.194511 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1419/2848] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 11:01:49.204011 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:49.205691 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:49.219725 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:49.264130 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cceda4-e381-4ff0-ade5-6d8c52c2164e admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:49.267702 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:49.268338 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:49.268617 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:49.268888 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:49.277929 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:49.279329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca126fad-0667-42a6-a71b-439fe44d66d0 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:49.289023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1430/2849] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 11:01:53.031842 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387580-a2c9-419e-981f-020db4abde0c None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.032378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387580-a2c9-419e-981f-020db4abde0c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.034688 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e387580-a2c9-419e-981f-020db4abde0c None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.036852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1420/2850] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 11:01:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 11:01:53.047636 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1776d-d389-412f-940f-7c836a419f3d None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.048181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1776d-d389-412f-940f-7c836a419f3d None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.048181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1776d-d389-412f-940f-7c836a419f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.107518 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-86c1776d-d389-412f-940f-7c836a419f3d None None] MFA Rules not processed for user `fd6ce0d979904b6e9db4968604e0192b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:53.108728 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-86c1776d-d389-412f-940f-7c836a419f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.122889 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1431/2851] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:53.135169 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387580-a2c9-419e-981f-020db4abde0c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.150814 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e387580-a2c9-419e-981f-020db4abde0c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.155019 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.155904 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.156362 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.156808 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.167167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.168913 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bb17e-816b-49c9-98cf-44df54279c9d service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.172576 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:53.196584 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1421/2852] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:53.271585 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.271992 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.272369 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.306672 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] MFA Rules not processed for user `88701ef1915d4ad2955a4bf31fe7634b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:53.308208 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.312204 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9acc-13ea-4995-9255-b8ac2eba1892 None None] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.312643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9acc-13ea-4995-9255-b8ac2eba1892 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.313045 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea9acc-13ea-4995-9255-b8ac2eba1892 None None] PATH_INFO: `/` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.314598 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1422/2853] 10.222.0.36 () {54 vars in 921 bytes} [Mon May 23 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 11:01:53.320665 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.321074 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.321464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.351592 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] MFA Rules not processed for user `8344a4e41ea548568e99784e3a6ef493`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:53.355065 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.400127 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1432/2854] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:53.405380 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.417845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86a111-1f15-46a9-b182-246a66ba0070 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.422784 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.423511 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.423860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.424290 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.425103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.427083 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.450932 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1423/2855] 10.222.0.36 () {58 vars in 1018 bytes} [Mon May 23 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 11:01:53.458492 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1433/2856] 10.222.0.36 () {60 vars in 1152 bytes} [Mon May 23 11:01:53 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 11:01:53.460171 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.463003 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.464883 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.473756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f9a3f0-2d28-48d0-841e-f2d79f2aef9f None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.553116 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d591bb7-b8bf-4cf9-bda4-95e9243617b4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.556878 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.557407 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.557690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.557991 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.572987 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.573799 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.574227 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.574579 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] PATH_INFO: `/v3/users` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.575350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.576346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:53.578067 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.599596 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1424/2857] 10.222.0.36 () {60 vars in 1143 bytes} [Mon May 23 11:01:53 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 11:01:53.604432 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.615380 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada95a06-b723-403c-aec8-e9e86a706d8a admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.621844 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.622337 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.622625 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.623216 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] PATH_INFO: `/v3/roles` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.623643 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.627378 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.638754 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1425/2858] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 11:01:53.642338 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.646215 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.647953 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.654506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f969-c4bb-4485-8312-435fa536f47b admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.662344 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1434/2859] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:53.662524 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.663829 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.663952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.664276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] PATH_INFO: `/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3/users/096dbdc229de48d08de8834c66bee240/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.680826 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21f4d52d4be4150b8aef28d6f2f9fa3, user_id=096dbdc229de48d08de8834c66bee240, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.683599 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.699397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1426/2860] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 11:01:53 2022] PUT /identity/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3/users/096dbdc229de48d08de8834c66bee240/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:01:53.704488 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.717068 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ef6eb-3765-40d6-9e20-7df0181dd24b service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.734506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.736268 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.787210 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:53.787595 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.788690 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.789019 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.789305 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.790106 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.791969 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.807745 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1435/2861] 10.222.0.36 () {58 vars in 1121 bytes} [Mon May 23 11:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 11:01:53.808819 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:53.814917 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82d71c-de35-4441-a7f6-906ce1693c2c admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.820322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0014a6-d499-4f00-9211-c79bb3509f2a admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:53.821233 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.821887 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.822206 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.822480 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.824757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:53.827329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:53.828241 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:53.828531 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:53.828814 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] PATH_INFO: `/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3/users/096dbdc229de48d08de8834c66bee240/roles/081e04042be64de0ac1c857efdae3324` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:53.830955 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21f4d52d4be4150b8aef28d6f2f9fa3, user_id=096dbdc229de48d08de8834c66bee240, role_id=081e04042be64de0ac1c857efdae3324)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.834270 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5d704-143b-48ab-8f20-2a4e44711bc4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.837634 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:53.839279 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:53.847098 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1436/2862] 10.222.0.36 () {60 vars in 1482 bytes} [Mon May 23 11:01:53 2022] PUT /identity/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3/users/096dbdc229de48d08de8834c66bee240/roles/081e04042be64de0ac1c857efdae3324 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:01:53.851395 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1427/2863] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:58.836255 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5b63-6b6f-46eb-b152-3e4ecf2a2d30 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:58.836696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5b63-6b6f-46eb-b152-3e4ecf2a2d30 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:58.836896 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e5b63-6b6f-46eb-b152-3e4ecf2a2d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:58.864486 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-d17e5b63-6b6f-46eb-b152-3e4ecf2a2d30 None None] MFA Rules not processed for user `096dbdc229de48d08de8834c66bee240`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:58.865786 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-d17e5b63-6b6f-46eb-b152-3e4ecf2a2d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:58.939409 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1437/2864] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:01:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:58.953148 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:58.954633 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:59.015918 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac4519-2bcd-4ce6-9bd6-57bc0ba9c95f service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:59.072757 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '069791ace6bc45a494dd77b732302466', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:59.073424 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:59.073714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:59.074021 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:59.138516 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:59.139975 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:59.149266 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1428/2865] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:01:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:01:59.253084 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018bd1-015d-4bf0-9fe7-9ba1642aa561 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:59.253385 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018bd1-015d-4bf0-9fe7-9ba1642aa561 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:59.253692 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0018bd1-015d-4bf0-9fe7-9ba1642aa561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:59.272312 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-e0018bd1-015d-4bf0-9fe7-9ba1642aa561 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:01:59.273805 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-e0018bd1-015d-4bf0-9fe7-9ba1642aa561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:59.282643 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:59.333418 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1438/2866] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 11:01:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:01:59.382069 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:01:59.383513 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:01:59.430573 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=100 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:01:59.474176 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99630ba3-22ae-4270-890e-f130542892f0 service glance] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:01:59.482116 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:01:59.482981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:01:59.483104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:01:59.483345 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:01:59.494201 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:01:59.496127 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4cc4d-d87b-458b-a491-2b84e928e4a4 service swift] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:01:59.506927 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1429/2867] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 11:01:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 11:02:08.366178 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:08.366723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:08.366723 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:08.384304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] MFA Rules not processed for user `88701ef1915d4ad2955a4bf31fe7634b`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:02:08.385572 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:08.392523 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1439/2868] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:02:09.504959 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=12 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:02:10.359658 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:10.359953 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:10.360235 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:10.378129 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:02:10.379914 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:10.387783 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1430/2869] 10.222.0.36 () {58 vars in 1009 bytes} [Mon May 23 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:02:10.400855 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:02:10.413320 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971105e-18e0-489f-aef3-3627970d6c06 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:02:10.418442 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:02:10.419199 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:10.419493 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:10.419775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:10.428113 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:02:10.429752 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:02:10.437851 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1440/2870] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 11:02:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 11:02:18.136415 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:02:18.192082 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f153da-0e4b-4034-9cc8-58e96d3417e8 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:02:18.196343 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:02:18.197027 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:18.197348 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:18.197641 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:18.199482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:18.210105 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:02:18.211547 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:02:18.219796 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1431/2871] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:02:18.917704 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:02:18.921045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:19.032053 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e946bee-7b04-4ce1-8ae3-1372be4410de service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:02:19.038670 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:02:19.039335 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:19.039625 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:19.039900 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:19.041715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:19.055247 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:02:19.056852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9035d5-54b2-4565-87cc-821fb9e7fc88 service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:02:19.069154 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1441/2872] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:02:28.216058 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=29 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:02:29.064291 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=62 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:02:36.494484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:02:36.503336 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c21608-3dba-4e97-9db7-cb20dbc7aa05 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:02:36.507089 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:02:36.507827 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:36.508181 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:36.508471 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:36.517581 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:02:36.519062 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:02:36.529433 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1432/2873] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:02:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:02:38.743458 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] REQUEST_METHOD: `POST` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:02:38.744087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:02:38.744087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:02:38.767243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.auth.core [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] MFA Rules not processed for user `5b53f05946e34f9f8395526212c5b17f`. Rule list: `[]` (Enabled: `True`). {{(pid=91697) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:02:38.769042 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:02:38.851080 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1442/2874] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:02:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 11:02:46.525154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:02:48.840892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=25 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:03:28.026441 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:28.035232 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff3dd-109f-471c-8f36-7aff986344ea service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:28.038839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:28.039467 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:28.039752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:28.040028 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:28.041791 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:28.100489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:28.102404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:28.111506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1433/2875] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:03:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:03:33.047999 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:33.067549 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201da88-d19f-4853-bb4b-da7854a2a4f3 None None] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:33.071669 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:33.072243 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:33.072522 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:33.072797 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:33.074567 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:33.164336 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:33.165775 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:33.175367 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1443/2876] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:03:36.131316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:36.133174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:36.279283 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9e7c-c0a7-41b3-b270-54f5392aa0a7 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:36.284391 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:36.285032 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:36.285315 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:36.285589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:36.287394 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:36.299519 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:36.301120 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:36.312077 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1434/2877] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:03:38.954804 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:38.965890 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300022e5-a822-4f1e-b0a1-6f959297d5d8 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:38.974451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:38.974451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:38.974451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:38.974451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:38.974863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:38.986447 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:38.988043 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:38.999715 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1444/2878] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:03:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 11:03:39.028984 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:39.038543 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:39.100276 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f3551-5428-4a7a-ad0c-3c68d70981c3 service placement] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:39.106540 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:39.107448 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:39.108314 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:39.108601 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] PATH_INFO: `/v3/users/43568344bfa54dd7a6b2cb5fe34422d7` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:39.109900 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43568344bfa54dd7a6b2cb5fe34422d7)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:39.113600 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:39.132464 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:03:39.172683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43568344bfa54dd7a6b2cb5fe34422d7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.186124 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43568344bfa54dd7a6b2cb5fe34422d7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.186957 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43568344bfa54dd7a6b2cb5fe34422d7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.187574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43568344bfa54dd7a6b2cb5fe34422d7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.200369 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43568344bfa54dd7a6b2cb5fe34422d7'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.214453 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1435/2879] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:03:39 2022] DELETE /identity/v3/users/43568344bfa54dd7a6b2cb5fe34422d7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 11:03:39.394507 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:39.405974 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6e56c-8853-47e4-b01d-6df644c81125 service placement] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:39.462980 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:39.463835 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:39.464119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:39.464392 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] PATH_INFO: `/v3/projects/c085ae456b46460ea86e4cf1dfd819c2` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:39.465565 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c085ae456b46460ea86e4cf1dfd819c2)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:39.467490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:39.509852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:03:39.514272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c085ae456b46460ea86e4cf1dfd819c2', 'request_id': 'req-28e671fd-7106-48e2-81fc-78a21be6b9ef'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.521880 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c085ae456b46460ea86e4cf1dfd819c2', 'request_id': 'req-28e671fd-7106-48e2-81fc-78a21be6b9ef'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.528235 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c085ae456b46460ea86e4cf1dfd819c2', 'request_id': 'req-28e671fd-7106-48e2-81fc-78a21be6b9ef'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.538667 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] The token cache is being invalidate because project c085ae456b46460ea86e4cf1dfd819c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:03:39.538954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.539644 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.541219 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1445/2880] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:03:39 2022] DELETE /identity/v3/projects/c085ae456b46460ea86e4cf1dfd819c2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:03:39.545152 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:39.546569 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:39.560163 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c055ba-8565-4ac5-b2eb-7dde3141c860 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:39.614385 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:39.615190 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:39.615489 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:39.615764 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] PATH_INFO: `/v3/users/1a18cd72a1334aad88e3bea937084262` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:39.616972 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a18cd72a1334aad88e3bea937084262)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:39.619484 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:39.670367 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a18cd72a1334aad88e3bea937084262'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.682682 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a18cd72a1334aad88e3bea937084262'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.683350 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a18cd72a1334aad88e3bea937084262'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.683952 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a18cd72a1334aad88e3bea937084262'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.691174 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d6f73d2e-06f0-4564-be77-37574e481806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a18cd72a1334aad88e3bea937084262'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:39.704003 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1436/2881] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:03:39 2022] DELETE /identity/v3/users/1a18cd72a1334aad88e3bea937084262 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:03:39.871512 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:39.873045 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:39.888310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e671fd-7106-48e2-81fc-78a21be6b9ef admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:39.946695 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:39.947562 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:39.947852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:39.948133 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] PATH_INFO: `/v3/projects/3b3bfe966a0344d2847514ec296dee76` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:39.949308 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3bfe966a0344d2847514ec296dee76)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:39.951275 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:39.995954 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3bfe966a0344d2847514ec296dee76', 'request_id': 'req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:40.002844 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3bfe966a0344d2847514ec296dee76', 'request_id': 'req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:40.008932 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3bfe966a0344d2847514ec296dee76', 'request_id': 'req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:40.019699 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] The token cache is being invalidate because project 3b3bfe966a0344d2847514ec296dee76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:03:40.020015 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:40.020696 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:03:40.022304 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1446/2882] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:03:39 2022] DELETE /identity/v3/projects/3b3bfe966a0344d2847514ec296dee76 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:03:49.700404 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=75,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:03:50.016490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:03:57.344334 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:57.344832 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:57.344946 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:57.371861 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] MFA Rules not processed for user `086bf8e27f4745c3a01adcfe8d617cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:03:57.373047 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:57.432776 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1437/2883] 10.222.0.36 () {58 vars in 967 bytes} [Mon May 23 11:03:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:03:57.472322 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:03:57.473711 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:03:57.543769 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0182ea9-4cc0-40c0-98c4-9ae9ff57c1d4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:03:57.548739 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef2bef098cce41279cc9a5d81f0dd782', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:03:57.549387 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:03:57.549674 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:03:57.549965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:03:57.558504 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:03:57.560087 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:03:57.568070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1447/2884] 10.222.0.36 () {60 vars in 1384 bytes} [Mon May 23 11:03:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 11:04:04.020482 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:04.021839 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:04.089015 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bb2d4-d431-4245-af74-e881726ec956 None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:04.093509 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:04.094397 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:04.094683 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:04.094958 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] PATH_INFO: `/v3/users/88701ef1915d4ad2955a4bf31fe7634b` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:04.096145 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] RBAC: Authorizing `identity:delete_user(user_id=88701ef1915d4ad2955a4bf31fe7634b)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:04.098199 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:04.143256 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88701ef1915d4ad2955a4bf31fe7634b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.155750 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88701ef1915d4ad2955a4bf31fe7634b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.156426 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88701ef1915d4ad2955a4bf31fe7634b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.157014 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88701ef1915d4ad2955a4bf31fe7634b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.164816 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88701ef1915d4ad2955a4bf31fe7634b'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.177167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1438/2885] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:04:04 2022] DELETE /identity/v3/users/88701ef1915d4ad2955a4bf31fe7634b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:04:04.350616 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:04.363882 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd99e3-58af-4f66-8ebd-60dfb4c9f941 service swift] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:04.420099 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:04.420948 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:04.421226 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:04.421498 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] PATH_INFO: `/v3/projects/e35c886cdaaf465eacd001e132128cbd` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:04.422680 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35c886cdaaf465eacd001e132128cbd)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:04.424577 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:04.469402 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35c886cdaaf465eacd001e132128cbd', 'request_id': 'req-dc6cd83b-05a3-4343-968b-8f9628caad96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.476875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35c886cdaaf465eacd001e132128cbd', 'request_id': 'req-dc6cd83b-05a3-4343-968b-8f9628caad96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.482938 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35c886cdaaf465eacd001e132128cbd', 'request_id': 'req-dc6cd83b-05a3-4343-968b-8f9628caad96'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.493346 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] The token cache is being invalidate because project e35c886cdaaf465eacd001e132128cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:04:04.493626 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.494310 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.495865 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1448/2886] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:04:04 2022] DELETE /identity/v3/projects/e35c886cdaaf465eacd001e132128cbd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:04:04.499867 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:04.501294 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:04.514285 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80447f10-6cbf-48ed-a7dd-141c40ad507d admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:04.518693 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:04:04.572104 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:04.572973 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:04.573258 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:04.573542 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] PATH_INFO: `/v3/users/096dbdc229de48d08de8834c66bee240` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:04.574756 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=096dbdc229de48d08de8834c66bee240)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:04.576681 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:04.621501 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096dbdc229de48d08de8834c66bee240'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.633502 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096dbdc229de48d08de8834c66bee240'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.634245 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096dbdc229de48d08de8834c66bee240'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.634852 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096dbdc229de48d08de8834c66bee240'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.641983 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-f653cb87-bea1-4a76-949c-0a772a9f3d5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096dbdc229de48d08de8834c66bee240'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.654840 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1439/2887] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:04:04 2022] DELETE /identity/v3/users/096dbdc229de48d08de8834c66bee240 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:04:04.821452 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:04.822897 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:04.835618 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cd83b-05a3-4343-968b-8f9628caad96 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:04.854649 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:04:04.895818 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:04.896684 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:04.896970 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:04.897248 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] PATH_INFO: `/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:04.899071 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21f4d52d4be4150b8aef28d6f2f9fa3)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:04.901254 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:04.946782 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21f4d52d4be4150b8aef28d6f2f9fa3', 'request_id': 'req-12074054-b669-41ee-ba25-8b8dd92387a5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.954220 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21f4d52d4be4150b8aef28d6f2f9fa3', 'request_id': 'req-12074054-b669-41ee-ba25-8b8dd92387a5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.960272 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21f4d52d4be4150b8aef28d6f2f9fa3', 'request_id': 'req-12074054-b669-41ee-ba25-8b8dd92387a5'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.970481 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] The token cache is being invalidate because project c21f4d52d4be4150b8aef28d6f2f9fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91697) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:04:04.970770 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.971398 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:04:04.972940 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1449/2888] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:04:04 2022] DELETE /identity/v3/projects/c21f4d52d4be4150b8aef28d6f2f9fa3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:04:14.651333 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:04:14.967536 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:04:46.586655 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] REQUEST_METHOD: `POST` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:46.587167 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:46.587273 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:46.604298 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.auth.core [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] MFA Rules not processed for user `5b53f05946e34f9f8395526212c5b17f`. Rule list: `[]` (Enabled: `True`). {{(pid=91696) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 11:04:46.605529 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:46.658368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1440/2889] 10.222.0.36 () {58 vars in 957 bytes} [Mon May 23 11:04:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 11:04:46.969054 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:46.970501 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:47.048860 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074054-b669-41ee-ba25-8b8dd92387a5 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:47.052416 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:47.053050 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:47.053329 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:47.054125 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:47.055852 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:47.064965 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:47.066417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:47.075076 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1450/2890] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:04:48.498809 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:48.500533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:48.596590 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a027a3-0686-4f5b-8095-569a5dcf631c None None] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:48.602293 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:48.602878 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:48.603161 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:48.603438 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:48.605211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:48.615775 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:48.618051 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:48.626879 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1441/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:04:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:04:50.672417 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:04:50.674023 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:50.766149 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef431-7b2f-4658-b900-5d7dd20d6f2a service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:04:50.771384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:04:50.772555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:04:50.772845 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:04:50.773146 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:04:50.775196 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:04:50.786726 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:04:50.789032 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:04:50.799090 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1451/2892] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:04:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:04:58.622436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=68 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:05:00.794698 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=76 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:05:05.433714 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:05:05.445211 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07213776-08ca-42a6-93f3-bc7a93c4eff0 service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:05:05.450945 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:05:05.451589 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:05:05.451876 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:05:05.452154 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:05:05.453929 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:05:05.467187 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:05:05.469095 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:05:05.480436 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1442/2893] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:05:10.255181 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:05:10.256663 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:05:10.272119 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5281a-56cb-47ba-be00-1957f2108239 service placement] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:05:10.277902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8344a4e41ea548568e99784e3a6ef493', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:05:10.278642 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:05:10.278994 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:05:10.279337 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:05:10.281136 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:05:10.293360 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:05:10.294963 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:05:10.306875 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1452/2894] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:05:15.475253 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:05:20.300979 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=6 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:06:39.850962 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:39.852368 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:39.864081 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08859063-4029-4553-87f3-0c13b5b3c682 service nova] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:39.868623 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:39.869272 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:39.869560 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:39.869841 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:39.871752 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:39.885676 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:39.887140 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:39.898793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1443/2895] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:06:41.085466 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:41.095921 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea30f0-16fc-468f-b5ad-b2869b10ab7e service nova] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:41.100201 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:41.100843 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:41.101126 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:41.101540 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:41.103384 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:41.118902 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:41.120767 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:41.133465 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1453/2896] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:06:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:06:49.893254 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=6 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:06:50.717994 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:50.726734 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946967fd-12da-4c7d-8d3f-4c5ae0eebcc6 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:50.730316 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd6ce0d979904b6e9db4968604e0192b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:50.730943 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] REQUEST_METHOD: `GET` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:50.731224 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:50.731500 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:50.733374 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:50.796883 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:50.798383 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:50.808475 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1444/2897] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:06:51.127451 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=6 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:06:56.929555 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:56.931391 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:56.946569 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e306984-c796-44ff-b870-a30f9bb33e00 service neutron] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:56.951070 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:56.951936 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:56.952217 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:56.952490 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] PATH_INFO: `/v3/users/5b53f05946e34f9f8395526212c5b17f` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:56.953666 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b53f05946e34f9f8395526212c5b17f)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:56.955757 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:57.001482 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b53f05946e34f9f8395526212c5b17f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.008785 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b53f05946e34f9f8395526212c5b17f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.009455 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b53f05946e34f9f8395526212c5b17f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.010103 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b53f05946e34f9f8395526212c5b17f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.017128 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b53f05946e34f9f8395526212c5b17f'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.034939 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1454/2898] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:06:56 2022] DELETE /identity/v3/users/5b53f05946e34f9f8395526212c5b17f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:06:57.211183 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:57.224613 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077bc7c-bcb5-48d9-b725-358c066eea20 service neutron] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:57.288533 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:57.289398 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:57.289723 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:57.289981 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] PATH_INFO: `/v3/projects/86e332b470c841db8d7d852c1c0a4a26` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:57.291184 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e332b470c841db8d7d852c1c0a4a26)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:57.293332 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:57.339358 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e332b470c841db8d7d852c1c0a4a26', 'request_id': 'req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.345329 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e332b470c841db8d7d852c1c0a4a26', 'request_id': 'req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.350782 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e332b470c841db8d7d852c1c0a4a26', 'request_id': 'req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.362574 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] The token cache is being invalidate because project 86e332b470c841db8d7d852c1c0a4a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:06:57.362859 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.363474 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.364997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1445/2899] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:06:57 2022] DELETE /identity/v3/projects/86e332b470c841db8d7d852c1c0a4a26 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:06:57.369497 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:57.370923 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:57.384366 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f582be-851e-4072-adc3-c06498981fe4 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:57.439759 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:57.440629 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:57.440907 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:57.441175 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] PATH_INFO: `/v3/users/d23769a34b2f420db1613c6b270b9117` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:57.442435 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23769a34b2f420db1613c6b270b9117)` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:57.444892 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:57.488869 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23769a34b2f420db1613c6b270b9117'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.496250 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23769a34b2f420db1613c6b270b9117'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.497059 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23769a34b2f420db1613c6b270b9117'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.497664 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23769a34b2f420db1613c6b270b9117'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.504866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.notifications [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23769a34b2f420db1613c6b270b9117'} {{(pid=91697) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.513863 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:06:57.527341 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1455/2900] 10.222.0.36 () {58 vars in 1223 bytes} [Mon May 23 11:06:57 2022] DELETE /identity/v3/users/d23769a34b2f420db1613c6b270b9117 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:06:57.696037 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Authenticating user token {{(pid=91696) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:06:57.697477 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.fernet_utils [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91696) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:06:57.710966 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffccb1-bbe2-4a32-a6db-ee4753094cf1 admin admin] Validating token access rules against request {{(pid=91696) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:06:57.731997 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:06:57.770004 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9753922dc94eb786d2155e45d2454a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6dd918cb8494a52af5c7b9bb5c60e89', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91696) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:06:57.770793 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:06:57.771071 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:06:57.771339 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] PATH_INFO: `/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8` {{(pid=91696) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:06:57.772506 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e2d9e0aa87a4791b8088a598f3e84b8)` {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:06:57.774450 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] RBAC: Authorization granted {{(pid=91696) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:06:57.820136 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e2d9e0aa87a4791b8088a598f3e84b8', 'request_id': 'req-22fd88bb-15cf-452f-b8d6-e4b1875891ff'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.826137 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e2d9e0aa87a4791b8088a598f3e84b8', 'request_id': 'req-22fd88bb-15cf-452f-b8d6-e4b1875891ff'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.831627 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e2d9e0aa87a4791b8088a598f3e84b8', 'request_id': 'req-22fd88bb-15cf-452f-b8d6-e4b1875891ff'} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.843238 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] The token cache is being invalidate because project 2e2d9e0aa87a4791b8088a598f3e84b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91696) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 11:06:57.843526 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.844157 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG keystone.notifications [None req-22fd88bb-15cf-452f-b8d6-e4b1875891ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91696) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 11:06:57.845685 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: [pid: 91696|app: 0|req: 1446/2901] 10.222.0.36 () {58 vars in 1232 bytes} [Mon May 23 11:06:57 2022] DELETE /identity/v3/projects/2e2d9e0aa87a4791b8088a598f3e84b8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 11:07:07.528156 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:07:07.840229 dv-cinder-836996-25-iscsi devstack@keystone.service[91696]: DEBUG dbcounter [-] [91696] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91696) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 11:09:55.369866 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Authenticating user token {{(pid=91697) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 11:09:55.371297 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:09:55.438898 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f56e-d05c-4cb1-93ea-d93b79e562b2 admin admin] Validating token access rules against request {{(pid=91697) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 11:09:55.496753 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c27854a0a16544cc9eab7c048fe03833', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52f7cf28d64e4d8f94b7da57a5afdcc8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91697) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 11:09:55.497333 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] REQUEST_METHOD: `GET` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 11:09:55.497612 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] SCRIPT_NAME: `/identity` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 11:09:55.497887 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91697) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 11:09:55.499678 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.fernet_utils [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91697) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 11:09:55.511062 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 11:09:55.512623 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ce5f1-9066-4f0f-af4b-18bf3b7a5f1b service placement] RBAC: Authorization granted {{(pid=91697) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 11:09:55.523016 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: [pid: 91697|app: 0|req: 1456/2902] 10.222.0.36 () {60 vars in 1365 bytes} [Mon May 23 11:09:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 11:10:05.518211 dv-cinder-836996-25-iscsi devstack@keystone.service[91697]: DEBUG dbcounter [-] [91697] Writing DB stats keystone:SELECT=58 {{(pid=91697) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}